[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:33 dist-test-slave-8hhm krb5kdc[19288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:33 dist-test-slave-8hhm krb5kdc[19288](info): set up 2 sockets
Aug 11 22:30:33 dist-test-slave-8hhm krb5kdc[19288](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 22:30:35.116966 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.456s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:35 dist-test-slave-8hhm krb5kdc[19288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:35.259125 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34515
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:34515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:35.387894 19313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:35.388219 19313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:35.388320 19313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:35.392431 19313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:35.392513 19313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:35.392532 19313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:35.392589 19313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:35.392629 19313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:35.398177 19313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:34515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:35.399410 19313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:35.400557 19313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:35.410451 19323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:35.410452 19320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:35.410451 19319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:35.410562 19313 server_base.cc:1034] running on GCE node
I20260811 22:30:35.411443 19313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:35.412719 19313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:35.413913 19313 hybrid_clock.cc:648] HybridClock initialized: now 1786487435413916 us; error 106 us; skew 500 ppm
Aug 11 22:30:35 dist-test-slave-8hhm krb5kdc[19288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487435, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:35.417454 19313 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:35.418519 19313 webserver.cc:460] Webserver started at http://127.18.211.190:39881/ using document root <none> and password file <none>
I20260811 22:30:35.419184 19313 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:35.419293 19313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:35.419575 19313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:35.421978 19313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "40213d3ecb1a8235fedbada9cd57b92e"
server_key_iv: "df3cdf1b034610d971ebb26f2e13a7af"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.422633 19313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "40213d3ecb1a8235fedbada9cd57b92e"
server_key_iv: "df3cdf1b034610d971ebb26f2e13a7af"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.426834 19313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:35.429594 19337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:35.430922 19313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:30:35.431087 19313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "40213d3ecb1a8235fedbada9cd57b92e"
server_key_iv: "df3cdf1b034610d971ebb26f2e13a7af"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.431217 19313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:35.444625 19313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:35.448258 19313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:35.448514 19313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:35.457535 19313 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:34515
I20260811 22:30:35.457557 19441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:34515 every 8 connection(s)
I20260811 22:30:35.458832 19313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:35.462114 19442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:35.467620 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19313
I20260811 22:30:35.467761 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:35.469177 19278 external_mini_cluster.cc:1428] Setting key 6a0b1714e130a81fd4f18783e77d9304
I20260811 22:30:35.469264 19442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb: Bootstrap starting.
I20260811 22:30:35.473116 19442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:35.474706 19442 log.cc:824] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:35.476729 19442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb: No bootstrap required, opened a new log
I20260811 22:30:35.482009 19442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34515 } }
I20260811 22:30:35.482342 19442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:35.482406 19442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf4fdd7dd6e4d22b0cf7104a9d392eb, State: Initialized, Role: FOLLOWER
I20260811 22:30:35.482929 19442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34515 } }
I20260811 22:30:35.483101 19442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:35.483193 19442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:35.483319 19442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:35.484333 19442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34515 } }
I20260811 22:30:35.484721 19442 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bf4fdd7dd6e4d22b0cf7104a9d392eb; no voters:
I20260811 22:30:35.485097 19442 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:35.485388 19450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:35.485774 19450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [term 1 LEADER]: Becoming Leader. State: Replica: 0bf4fdd7dd6e4d22b0cf7104a9d392eb, State: Running, Role: LEADER
I20260811 22:30:35.486179 19450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34515 } }
I20260811 22:30:35.486619 19442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:35.488778 19454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34515 } } }
I20260811 22:30:35.488922 19454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:35.489482 19455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf4fdd7dd6e4d22b0cf7104a9d392eb. Latest consensus state: current_term: 1 leader_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf4fdd7dd6e4d22b0cf7104a9d392eb" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34515 } } }
I20260811 22:30:35.489598 19455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:35.490020 19464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:35.493971 19464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 22:30:35 dist-test-slave-8hhm krb5kdc[19288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:35.502007 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:35.472441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.18.211.190:34515)
0811 22:30:35.473000 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:35.473048 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:35.474298 (+ 1250us) server_negotiation.cc:407] Connection header received
0811 22:30:35.481547 (+ 7249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:35.481575 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:35.481970 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:35.482443 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:35.485920 (+ 3477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:35.487854 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:35.489807 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:35.490610 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:35.490786 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:35.495449 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:35.495455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:35.495468 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:35.495502 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:35.497844 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:35.498595 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:35.498602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:35.498608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:35.498768 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:35.499287 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:35.499290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:35.499291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:35.499807 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:35.499969 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:35.500744 (+ 775us) server_negotiation.cc:299] Negotiation successful
0811 22:30:35.500989 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":160,"threads_started":1}
I20260811 22:30:35.503003 19464 catalog_manager.cc:1347] Generated new cluster ID: 18333dafcc6b4fbb8c8e4087af223ba4
I20260811 22:30:35.503147 19464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:35.516449 19464 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:35.517503 19464 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:35.530046 19464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bf4fdd7dd6e4d22b0cf7104a9d392eb: Generated new TSK 0
I20260811 22:30:35.530833 19464 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:30:35.569139 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34515
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:35.711694 19484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:35.712016 19484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:35.712121 19484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:35.716312 19484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:35.716396 19484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:35.716526 19484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:30:35.722093 19484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:35.723294 19484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:35.724406 19484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:35.735507 19494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:35.735498 19491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:35.735566 19492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:35.735816 19484 server_base.cc:1034] running on GCE node
I20260811 22:30:35.736377 19484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:35.737104 19484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:35.738281 19484 hybrid_clock.cc:648] HybridClock initialized: now 1786487435738245 us; error 47 us; skew 500 ppm
Aug 11 22:30:35 dist-test-slave-8hhm krb5kdc[19288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487435, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:35.741676 19484 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:30:35.742825 19484 webserver.cc:460] Webserver started at http://127.18.211.129:35385/ using document root <none> and password file <none>
I20260811 22:30:35.743489 19484 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:35.743562 19484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:35.743831 19484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:35.746343 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "5f2a42f86b3346dfbe308d9ee10312f7"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "8802bd6c6365dfd1a930092c45f2bf6f"
server_key_iv: "411a6d240ab9db3a24b0ab07a9850d5c"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.747021 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "5f2a42f86b3346dfbe308d9ee10312f7"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "8802bd6c6365dfd1a930092c45f2bf6f"
server_key_iv: "411a6d240ab9db3a24b0ab07a9850d5c"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.751471 19484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:35.754334 19505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:35.755689 19484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:30:35.755832 19484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "5f2a42f86b3346dfbe308d9ee10312f7"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "8802bd6c6365dfd1a930092c45f2bf6f"
server_key_iv: "411a6d240ab9db3a24b0ab07a9850d5c"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.755956 19484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:35.768501 19484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:35.771780 19484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:35.772010 19484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:35.772687 19484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:35.773658 19484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:35.773707 19484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:35.773780 19484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:35.773813 19484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:35.784358 19484 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:45643
I20260811 22:30:35.784390 19693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:45643 every 8 connection(s)
I20260811 22:30:35.785454 19484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:30:35.792570 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19484
I20260811 22:30:35.792670 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:30:35.792995 19278 external_mini_cluster.cc:1428] Setting key a2289746494ff5fb831a23066fd89545
Aug 11 22:30:35 dist-test-slave-8hhm krb5kdc[19288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487435, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:35.802548 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:35.787939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:49551 (local address 127.18.211.190:34515)
0811 22:30:35.788121 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:35.788126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:35.789233 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 22:30:35.790709 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:35.790712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:35.790766 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:35.790853 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:35.792161 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:35.792905 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:35.794375 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:35.794697 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:35.794820 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:35.798060 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:35.798064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:35.798071 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:35.798098 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:35.799904 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:35.800489 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:35.800495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:35.800500 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:35.800608 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:35.801053 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:35.801056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:35.801058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:35.801406 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:35.801528 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:35.802124 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 22:30:35.802289 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:30:35.803297 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:35.788420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:34515 (local address 127.18.211.129:49551)
0811 22:30:35.789028 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:35.789074 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:35.790463 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:35.791030 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:35.791037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:35.791441 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:35.791966 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:35.791980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:35.793088 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:35.793092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:35.794241 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:35.794253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:35.794898 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:35.794902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:35.795114 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:35.795832 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:35.795860 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:35.797850 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:35.800043 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:35.800050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:35.800065 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:35.800356 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:35.800749 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:35.800752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:35.800754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:35.800918 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:35.801527 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:35.801534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:35.801825 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 22:30:35.801997 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 22:30:35.802298 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":182,"threads_started":1}
I20260811 22:30:35.804728 19694 heartbeater.cc:347] Connected to a master server at 127.18.211.190:34515
I20260811 22:30:35.804997 19694 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:35.805578 19694 heartbeater.cc:511] Master 127.18.211.190:34515 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:30:35.807333 19382 ts_manager.cc:194] Registered new tserver with Master: 5f2a42f86b3346dfbe308d9ee10312f7 (127.18.211.129:45643)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:30:35.808732 19382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:49551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:30:35.829005 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34515
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:35.949306 19700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:35.949592 19700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:35.949680 19700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:35.954439 19700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:35.954502 19700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:35.954596 19700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:30:35.960041 19700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:35.961190 19700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:35.962329 19700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:35.972563 19706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:35.972563 19709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:35.972577 19707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:35.972868 19700 server_base.cc:1034] running on GCE node
I20260811 22:30:35.973402 19700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:35.974152 19700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:35.975332 19700 hybrid_clock.cc:648] HybridClock initialized: now 1786487435975312 us; error 29 us; skew 500 ppm
Aug 11 22:30:35 dist-test-slave-8hhm krb5kdc[19288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487435, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:35.978601 19700 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:30:35.979561 19700 webserver.cc:460] Webserver started at http://127.18.211.130:37517/ using document root <none> and password file <none>
I20260811 22:30:35.980167 19700 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:35.980289 19700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:35.980545 19700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:35.982744 19700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "2312c8fc52734df0abf186f398290d8d"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "ea4137a42fd072c5ee5ca1c3cd576cd0"
server_key_iv: "fa205bdc63c3e1550cccdc4d2b82d6d1"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.983345 19700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "2312c8fc52734df0abf186f398290d8d"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "ea4137a42fd072c5ee5ca1c3cd576cd0"
server_key_iv: "fa205bdc63c3e1550cccdc4d2b82d6d1"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.987581 19700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:35.990593 19729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:35.991871 19700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:35.992024 19700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "2312c8fc52734df0abf186f398290d8d"
format_stamp: "Formatted at 2026-08-11 22:30:35 on dist-test-slave-8hhm"
server_key: "ea4137a42fd072c5ee5ca1c3cd576cd0"
server_key_iv: "fa205bdc63c3e1550cccdc4d2b82d6d1"
server_key_version: "encryptionkey@0"
I20260811 22:30:35.992154 19700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:36.026223 19700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:36.029867 19700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:36.030092 19700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:36.030730 19700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:36.031606 19700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:36.031682 19700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:36.031767 19700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:36.031812 19700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:36.042775 19700 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:36035
I20260811 22:30:36.042810 19925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:36035 every 8 connection(s)
I20260811 22:30:36.043730 19700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:30:36.047626 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19700
I20260811 22:30:36.047711 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:30:36.047964 19278 external_mini_cluster.cc:1428] Setting key c06b1d8e05fa58efc4768be9e77d46fa
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[19288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487435, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:36.060215 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:36.045792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:57011 (local address 127.18.211.190:34515)
0811 22:30:36.045939 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:36.045943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:36.046834 (+ 891us) server_negotiation.cc:407] Connection header received
0811 22:30:36.048303 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:36.048308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:36.048368 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:36.048476 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:36.049998 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.050514 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.052111 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.052334 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.052423 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:36.055717 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:36.055724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:36.055730 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:36.055767 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:36.057666 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.058254 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.058259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.058262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.058366 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.058859 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.058864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.058866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.059128 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:36.059268 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:36.059883 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 22:30:36.060029 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:30:36.061066 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:36.046209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:34515 (local address 127.18.211.130:57011)
0811 22:30:36.046658 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:36.046717 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:36.048083 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:36.048694 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:36.048704 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:36.049193 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:36.049845 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:36.049861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.050648 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.050651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:36.051970 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:36.051985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.052498 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.052503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:36.052748 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:36.053491 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:36.053508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:36.055481 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:36.057801 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:36.057806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:36.057820 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:36.058130 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:36.058511 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:36.058514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:36.058515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:36.058724 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:36.059288 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:36.059296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:36.059603 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 22:30:36.059778 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 22:30:36.060054 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260811 22:30:36.062345 19928 heartbeater.cc:347] Connected to a master server at 127.18.211.190:34515
I20260811 22:30:36.062637 19928 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:36.063218 19928 heartbeater.cc:511] Master 127.18.211.190:34515 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:30:36.064594 19382 ts_manager.cc:194] Registered new tserver with Master: 2312c8fc52734df0abf186f398290d8d (127.18.211.130:36035)
I20260811 22:30:36.065585 19382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:57011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:30:36.080351 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34515
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:36.200157 19936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:36.200443 19936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:36.200488 19936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:36.204712 19936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:36.204789 19936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:36.204854 19936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:30:36.210290 19936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:36.211391 19936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:36.212405 19936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:36.222566 19946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:36.222621 19944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:36.222563 19943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:36.222836 19936 server_base.cc:1034] running on GCE node
I20260811 22:30:36.223325 19936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:36.223987 19936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:36.225257 19936 hybrid_clock.cc:648] HybridClock initialized: now 1786487436225172 us; error 53 us; skew 500 ppm
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[19288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:36.228322 19936 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:30:36.229260 19936 webserver.cc:460] Webserver started at http://127.18.211.131:39777/ using document root <none> and password file <none>
I20260811 22:30:36.229830 19936 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:36.229887 19936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:36.230065 19936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:36.232120 19936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "0e5c2b08ce9e4b0eb70baf7ebe8cc5bc"
format_stamp: "Formatted at 2026-08-11 22:30:36 on dist-test-slave-8hhm"
server_key: "d5dc0390d7bd1d0ab3ca01e93034a997"
server_key_iv: "625087c1e43c5a38556d624023277921"
server_key_version: "encryptionkey@0"
I20260811 22:30:36.232649 19936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "0e5c2b08ce9e4b0eb70baf7ebe8cc5bc"
format_stamp: "Formatted at 2026-08-11 22:30:36 on dist-test-slave-8hhm"
server_key: "d5dc0390d7bd1d0ab3ca01e93034a997"
server_key_iv: "625087c1e43c5a38556d624023277921"
server_key_version: "encryptionkey@0"
I20260811 22:30:36.237267 19936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:30:36.240489 19957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:36.242054 19936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:36.242223 19936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "0e5c2b08ce9e4b0eb70baf7ebe8cc5bc"
format_stamp: "Formatted at 2026-08-11 22:30:36 on dist-test-slave-8hhm"
server_key: "d5dc0390d7bd1d0ab3ca01e93034a997"
server_key_iv: "625087c1e43c5a38556d624023277921"
server_key_version: "encryptionkey@0"
I20260811 22:30:36.242358 19936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:36.264550 19936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:36.268126 19936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:36.268356 19936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:36.268998 19936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:36.269918 19936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:36.269995 19936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:36.270067 19936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:36.270115 19936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:36.280313 19936 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:42801
I20260811 22:30:36.280340 20174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:42801 every 8 connection(s)
I20260811 22:30:36.281339 19936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:30:36.288422 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19936
I20260811 22:30:36.288574 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:30:36.288857 19278 external_mini_cluster.cc:1428] Setting key fff629bafd97372099e02bc31a1e83bd
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[19288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:36.296294 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:36.283301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:34415 (local address 127.18.211.190:34515)
0811 22:30:36.283443 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:36.283447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:36.284348 (+ 901us) server_negotiation.cc:407] Connection header received
0811 22:30:36.285688 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:36.285691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:36.285740 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:36.285818 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:36.287048 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.287651 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.289020 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.289334 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.289485 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:36.292736 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:36.292739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:36.292742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:36.292769 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:36.294352 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.294837 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.294840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.294841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.294915 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.295234 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.295236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.295237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.295418 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:36.295536 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:36.296059 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 22:30:36.296166 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:30:36.297300 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:36.283715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:34515 (local address 127.18.211.131:34415)
0811 22:30:36.284183 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:36.284227 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:36.285491 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:36.285989 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:36.285995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:36.286412 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:36.286914 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:36.286928 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.287766 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.287768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:36.288885 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:36.288895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.289704 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.289707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:36.289936 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:36.290664 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:36.290689 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:36.292562 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:36.294469 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:36.294475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:36.294489 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:36.294765 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:36.294995 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:36.294997 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:36.294999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:36.295139 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:36.295555 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:36.295563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:36.295835 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 22:30:36.295999 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:30:36.296279 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260811 22:30:36.298496 20177 heartbeater.cc:347] Connected to a master server at 127.18.211.190:34515
I20260811 22:30:36.298741 20177 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:36.299238 20177 heartbeater.cc:511] Master 127.18.211.190:34515 requested a full tablet report, sending...
I20260811 22:30:36.300314 19382 ts_manager.cc:194] Registered new tserver with Master: 0e5c2b08ce9e4b0eb70baf7ebe8cc5bc (127.18.211.131:42801)
I20260811 22:30:36.300902 19382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:34415
I20260811 22:30:36.304155 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[19288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487436, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[19288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487436, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:30:36.333145 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:36.323230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.18.211.129:45643)
0811 22:30:36.323578 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:36.323587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:36.323615 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:30:36.323724 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:36.323736 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:36.323936 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:36.324107 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:36.324839 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.325693 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.326553 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.326946 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.327081 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:36.329638 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:36.329648 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:36.329664 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:36.329696 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:36.331259 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.331739 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.331747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.331750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.331857 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.332208 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.332210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.332211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.332464 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:36.332637 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:36.332846 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 22:30:36.332980 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":122,"threads_started":1}
W20260811 22:30:36.334230 19551 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49138
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[19288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[19288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:30:36.364653 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:36.355763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.18.211.129:45643)
0811 22:30:36.356042 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:36.356046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:36.356066 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:30:36.356182 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:36.356184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:36.356235 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:36.356320 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:36.357050 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.357674 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.358531 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:36.358761 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:36.358856 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:36.361567 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:36.361571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:36.361573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:36.361602 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:36.363136 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.363516 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.363520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.363522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.363610 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:36.363848 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:36.363850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:36.363851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:36.364012 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:36.364136 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:36.364334 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 22:30:36.364435 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 22:30:36.365870 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19484
I20260811 22:30:36.385263 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19700
I20260811 22:30:36.406337 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19936
I20260811 22:30:36.430996 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19313
2026-08-11T22:30:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2819 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[20206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[20206](info): set up 2 sockets
Aug 11 22:30:36 dist-test-slave-8hhm krb5kdc[20206](info): commencing operation
krb5kdc: starting...
W20260811 22:30:37.773134 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:37 dist-test-slave-8hhm krb5kdc[20206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:37.908775 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45081
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:38.024504 20228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:38.024785 20228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:38.024825 20228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:38.028848 20228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:38.028911 20228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:38.028930 20228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:38.028947 20228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:38.028962 20228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:38.034389 20228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:38.035497 20228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:38.036564 20228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:38.045287 20239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:38.045287 20237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:38.045322 20241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:38.045617 20228 server_base.cc:1034] running on GCE node
I20260811 22:30:38.046241 20228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:38.047380 20228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:38.048602 20228 hybrid_clock.cc:648] HybridClock initialized: now 1786487438048588 us; error 28 us; skew 500 ppm
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:38.051569 20228 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:38.052567 20228 webserver.cc:460] Webserver started at http://127.18.211.190:37287/ using document root <none> and password file <none>
I20260811 22:30:38.053184 20228 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:38.053333 20228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:38.053587 20228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:38.055766 20228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "bc61defc1ffb419c92e49ed81a5a03bf"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "7f69263d4ebc14b9e830f4312898e2dd"
server_key_iv: "3fb162728671d47584a17778dc31c608"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.056376 20228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "bc61defc1ffb419c92e49ed81a5a03bf"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "7f69263d4ebc14b9e830f4312898e2dd"
server_key_iv: "3fb162728671d47584a17778dc31c608"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.060372 20228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 22:30:38.062917 20252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.064200 20228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:30:38.064348 20228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "bc61defc1ffb419c92e49ed81a5a03bf"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "7f69263d4ebc14b9e830f4312898e2dd"
server_key_iv: "3fb162728671d47584a17778dc31c608"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.064472 20228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:38.087052 20228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:38.090158 20228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:38.090389 20228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:38.098594 20228 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45081
I20260811 22:30:38.098626 20347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45081 every 8 connection(s)
I20260811 22:30:38.099638 20228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:38.102617 20349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:38.106746 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20228
I20260811 22:30:38.106841 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:38.107115 19278 external_mini_cluster.cc:1428] Setting key 55430c1764963e93c21ade1b02b2c8f7
I20260811 22:30:38.108861 20349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf: Bootstrap starting.
I20260811 22:30:38.111542 20349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:38.112337 20349 log.cc:824] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:38.114658 20349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf: No bootstrap required, opened a new log
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:38.118876 20349 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45081 } }
I20260811 22:30:38.119112 20349 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:38.119194 20349 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc61defc1ffb419c92e49ed81a5a03bf, State: Initialized, Role: FOLLOWER
I20260811 22:30:38.119714 20349 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45081 } }
I20260811 22:30:38.119897 20349 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:38.119973 20349 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:38.120072 20349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:38.121287 20349 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45081 } }
I20260811 22:30:38.121685 20349 leader_election.cc:302] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc61defc1ffb419c92e49ed81a5a03bf; no voters:
I20260811 22:30:38.122053 20349 leader_election.cc:288] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:38.122184 20356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:38.122450 20356 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [term 1 LEADER]: Becoming Leader. State: Replica: bc61defc1ffb419c92e49ed81a5a03bf, State: Running, Role: LEADER
I20260811 22:30:38.122807 20356 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45081 } }
I20260811 22:30:38.123406 20349 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:38.124280 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.108703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40896 (local address 127.18.211.190:45081)
0811 22:30:38.109137 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.109160 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.109198 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:30:38.109931 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.109948 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.110296 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.110594 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:38.111523 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.112682 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.113823 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.114281 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.114410 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.117395 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:38.117404 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:38.117425 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:38.117467 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:38.120554 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.121017 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.121023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.121029 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.121193 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.121580 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.121584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.121585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.122193 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:38.122414 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.122871 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.123181 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":143,"threads_started":1}
I20260811 22:30:38.124610 20357 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45081 } } }
I20260811 22:30:38.124758 20357 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:38.124984 20358 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc61defc1ffb419c92e49ed81a5a03bf. Latest consensus state: current_term: 1 leader_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc61defc1ffb419c92e49ed81a5a03bf" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45081 } } }
I20260811 22:30:38.125077 20358 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:38.125360 20367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:38.129112 20367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:38.135447 20367 catalog_manager.cc:1347] Generated new cluster ID: 50e931a082aa40b68a262da52a049d19
I20260811 22:30:38.135540 20367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:38.153352 20367 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:38.154719 20367 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:38.166222 20367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc61defc1ffb419c92e49ed81a5a03bf: Generated new TSK 0
I20260811 22:30:38.167145 20367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:30:38.198698 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45081
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:38.317301 20392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:38.317592 20392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:38.317674 20392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:38.321746 20392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:38.321813 20392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:38.321934 20392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:30:38.327505 20392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45081
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:38.328651 20392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:38.329751 20392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:38.339596 20400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:38.339605 20402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:38.339653 20392 server_base.cc:1034] running on GCE node
W20260811 22:30:38.339867 20404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:38.340240 20392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:38.340906 20392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:38.342110 20392 hybrid_clock.cc:648] HybridClock initialized: now 1786487438342076 us; error 51 us; skew 500 ppm
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:38.345006 20392 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:30:38.346002 20392 webserver.cc:460] Webserver started at http://127.18.211.129:35481/ using document root <none> and password file <none>
I20260811 22:30:38.346593 20392 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:38.346657 20392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:38.346899 20392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:38.348939 20392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "39b9a5df5a054ee1a4897278100fa5aa"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "e3fbe5786643297a8304d5484ddc6507"
server_key_iv: "6f4e1f43684fa7d2245208ace08a45f2"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.349566 20392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "39b9a5df5a054ee1a4897278100fa5aa"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "e3fbe5786643297a8304d5484ddc6507"
server_key_iv: "6f4e1f43684fa7d2245208ace08a45f2"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.353360 20392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:30:38.355897 20416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.357161 20392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:30:38.357596 20392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "39b9a5df5a054ee1a4897278100fa5aa"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "e3fbe5786643297a8304d5484ddc6507"
server_key_iv: "6f4e1f43684fa7d2245208ace08a45f2"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.357728 20392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:38.368719 20392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:38.372313 20392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:38.372540 20392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:38.373163 20392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:38.374061 20392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:38.374136 20392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.374205 20392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:38.374254 20392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.383163 20392 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:38195
I20260811 22:30:38.383190 20615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:38195 every 8 connection(s)
I20260811 22:30:38.384095 20392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:30:38.385537 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20392
I20260811 22:30:38.385646 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:30:38.385949 19278 external_mini_cluster.cc:1428] Setting key c9d1cf524c690350a92eff6267f64f2d
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:30:38.399267 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.386047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:48465 (local address 127.18.211.190:45081)
0811 22:30:38.386177 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.386181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.387127 (+ 946us) server_negotiation.cc:407] Connection header received
0811 22:30:38.388283 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.388285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.388327 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.388400 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:38.390081 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.390929 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.392081 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.392347 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.392431 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.395429 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:38.395433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:38.395434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:38.395458 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:38.396940 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.397597 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.397603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.397605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.397692 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.398058 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.398061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.398062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.398296 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:38.398406 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.398936 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.399078 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:30:38.400084 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.386484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45081 (local address 127.18.211.129:48465)
0811 22:30:38.386983 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:38.387026 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:38.388045 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:38.388589 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:38.388601 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:38.389143 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:38.389913 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.389940 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.391064 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.391066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.391963 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.391972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.392524 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.392531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.392767 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.393468 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:38.393494 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:38.395252 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:38.397107 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:38.397114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:38.397131 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:38.397483 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:38.397790 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:38.397793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:38.397795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:38.397946 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:38.398406 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:38.398414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:38.398682 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 22:30:38.398878 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:30:38.399240 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:30:38.401458 20617 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45081
I20260811 22:30:38.401759 20617 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:38.402395 20617 heartbeater.cc:511] Master 127.18.211.190:45081 requested a full tablet report, sending...
I20260811 22:30:38.404148 20287 ts_manager.cc:194] Registered new tserver with Master: 39b9a5df5a054ee1a4897278100fa5aa (127.18.211.129:38195)
I20260811 22:30:38.405436 20287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:48465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:30:38.420425 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45081
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:38.539220 20625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:38.539502 20625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:38.539548 20625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:38.543888 20625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:38.543972 20625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:38.544045 20625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:30:38.549911 20625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45081
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:38.551049 20625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:38.552124 20625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:38.562525 20636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:38.562500 20631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:38.562469 20633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:38.562702 20625 server_base.cc:1034] running on GCE node
I20260811 22:30:38.563274 20625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:38.564002 20625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:38.565284 20625 hybrid_clock.cc:648] HybridClock initialized: now 1786487438565241 us; error 51 us; skew 500 ppm
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:38.574545 20625 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:30:38.575654 20625 webserver.cc:460] Webserver started at http://127.18.211.130:35021/ using document root <none> and password file <none>
I20260811 22:30:38.576329 20625 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:38.576440 20625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:38.576715 20625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:38.579087 20625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "4b21b8bb75ac4a19992081eb29c34a93"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "0e7456b0e940e8ec97a931da4d7ff6e6"
server_key_iv: "ffbbc1c36cca41a1e4bccd091b75296d"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.579704 20625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "4b21b8bb75ac4a19992081eb29c34a93"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "0e7456b0e940e8ec97a931da4d7ff6e6"
server_key_iv: "ffbbc1c36cca41a1e4bccd091b75296d"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.583806 20625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:38.586431 20658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.587668 20625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:30:38.587815 20625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "4b21b8bb75ac4a19992081eb29c34a93"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "0e7456b0e940e8ec97a931da4d7ff6e6"
server_key_iv: "ffbbc1c36cca41a1e4bccd091b75296d"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.587939 20625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:38.604918 20625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:38.608795 20625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:38.609030 20625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:38.609766 20625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:38.610657 20625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:38.610733 20625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.610803 20625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:38.610848 20625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.620151 20625 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:39645
I20260811 22:30:38.620175 20857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:39645 every 8 connection(s)
I20260811 22:30:38.621474 20625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:30:38.629340 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20625
I20260811 22:30:38.629455 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:30:38.629755 19278 external_mini_cluster.cc:1428] Setting key 245e7c9ac36ac2c6bd831bf06755dccc
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:38.636924 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.623673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:58253 (local address 127.18.211.190:45081)
0811 22:30:38.623849 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.623854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.624954 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 22:30:38.626015 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.626019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.626081 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.626193 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:38.627383 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.627914 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.629112 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.629506 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.629617 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.632599 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:38.632605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:38.632612 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:38.632647 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:38.634279 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.634965 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.634968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.634970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.635104 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.635535 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.635541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.635545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.635826 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:38.635937 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.636487 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.636659 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:30:38.637646 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.624227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45081 (local address 127.18.211.130:58253)
0811 22:30:38.624788 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:38.624835 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:38.625778 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:38.626367 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:38.626374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:38.626744 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:38.627248 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.627262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.628024 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.628026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.628909 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.628919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.629665 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.629668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.629869 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.630698 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:38.630719 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:38.632419 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:38.634406 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:38.634416 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:38.634436 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:38.634849 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:38.635258 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:38.635263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:38.635265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:38.635430 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:38.635937 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:38.635942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:38.636215 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 22:30:38.636386 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 22:30:38.636669 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260811 22:30:38.638967 20858 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45081
I20260811 22:30:38.639225 20858 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:38.639758 20858 heartbeater.cc:511] Master 127.18.211.190:45081 requested a full tablet report, sending...
I20260811 22:30:38.640919 20287 ts_manager.cc:194] Registered new tserver with Master: 4b21b8bb75ac4a19992081eb29c34a93 (127.18.211.130:39645)
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:30:38.641619 20287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:58253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:30:38.661072 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45081
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:38.780113 20869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:38.780396 20869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:38.780442 20869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:38.784616 20869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:38.784694 20869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:38.784755 20869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:30:38.790316 20869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45081
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:38.791579 20869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:38.792721 20869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:38.803486 20881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:38.803700 20885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:38.803486 20879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:38.803725 20869 server_base.cc:1034] running on GCE node
I20260811 22:30:38.804314 20869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:38.804935 20869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:38.806131 20869 hybrid_clock.cc:648] HybridClock initialized: now 1786487438806099 us; error 38 us; skew 500 ppm
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:38.809485 20869 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:30:38.810495 20869 webserver.cc:460] Webserver started at http://127.18.211.131:42919/ using document root <none> and password file <none>
I20260811 22:30:38.811144 20869 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:38.811213 20869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:38.811477 20869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:38.813797 20869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "31c6dc306e4c461db9b93279d4e31195"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "49fc8725b447d18be1e8703a5c3028c9"
server_key_iv: "cbde140f5777d84c9ae0ef9ae18e2082"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.814458 20869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "31c6dc306e4c461db9b93279d4e31195"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "49fc8725b447d18be1e8703a5c3028c9"
server_key_iv: "cbde140f5777d84c9ae0ef9ae18e2082"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.818780 20869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:38.821612 20894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.823225 20869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:30:38.823398 20869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "31c6dc306e4c461db9b93279d4e31195"
format_stamp: "Formatted at 2026-08-11 22:30:38 on dist-test-slave-8hhm"
server_key: "49fc8725b447d18be1e8703a5c3028c9"
server_key_iv: "cbde140f5777d84c9ae0ef9ae18e2082"
server_key_version: "encryptionkey@0"
I20260811 22:30:38.823639 20869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:38.846169 20869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:38.849659 20869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:38.849833 20869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:38.850421 20869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:38.851256 20869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:38.851306 20869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.851346 20869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:38.851362 20869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:38.861574 20869 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:40505
I20260811 22:30:38.861642 21110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:40505 every 8 connection(s)
I20260811 22:30:38.862673 20869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:30:38.869988 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20869
I20260811 22:30:38.870095 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:30:38.870405 19278 external_mini_cluster.cc:1428] Setting key 63d6ad0f9e6dfba1cbc25a10761a02e3
Aug 11 22:30:38 dist-test-slave-8hhm krb5kdc[20206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:38.877167 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.864717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:49713 (local address 127.18.211.190:45081)
0811 22:30:38.864867 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.864871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.865844 (+ 973us) server_negotiation.cc:407] Connection header received
0811 22:30:38.866823 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.866827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.866885 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.866996 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:38.868291 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.868828 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.870175 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.870506 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.870632 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.873746 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:38.873750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:38.873752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:38.873781 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:38.875124 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.875624 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.875627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.875628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.875707 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.876054 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.876056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.876057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.876241 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:38.876319 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.876903 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.877041 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:30:38.878018 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.865109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45081 (local address 127.18.211.131:49713)
0811 22:30:38.865692 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:38.865738 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:38.866600 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:38.867186 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:38.867194 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:38.867620 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:38.868123 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.868137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.868966 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.868969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.870012 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.870023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.870671 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.870674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.870899 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.871602 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:38.871623 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:38.873550 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:38.875237 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:38.875243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:38.875256 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:38.875523 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:38.875787 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:38.875789 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:38.875790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:38.875932 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:38.876346 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:38.876352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:38.876625 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 22:30:38.876798 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 22:30:38.877077 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":124,"threads_started":1}
I20260811 22:30:38.879277 21111 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45081
I20260811 22:30:38.879580 21111 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:38.880119 21111 heartbeater.cc:511] Master 127.18.211.190:45081 requested a full tablet report, sending...
I20260811 22:30:38.881340 20287 ts_manager.cc:194] Registered new tserver with Master: 31c6dc306e4c461db9b93279d4e31195 (127.18.211.131:40505)
I20260811 22:30:38.881925 20287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:49713
I20260811 22:30:38.891462 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:30:38.904311 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.896527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40904 (local address 127.18.211.190:45081)
0811 22:30:38.896681 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.896688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.896901 (+ 213us) server_negotiation.cc:407] Connection header received
0811 22:30:38.897060 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.897063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.897113 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.897195 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:38.898017 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.898537 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.899384 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.899593 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.899682 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.901332 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:38.901335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:38.901337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:38.901359 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:38.902651 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.903057 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.903059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.903060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.903142 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:38.903401 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:38.903403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:38.903404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:38.903626 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:38.903743 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.903947 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.904107 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:30:38.909773 20287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:30:38.912901 20287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:38.928968 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.922128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56140 (local address 127.18.211.129:38195)
0811 22:30:38.922435 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.922442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.922538 (+ 96us) server_negotiation.cc:407] Connection header received
0811 22:30:38.922815 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.922822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.923090 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.923256 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:38.924294 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.925585 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.927409 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.928184 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.928310 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.928445 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.928667 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.928794 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":74,"threads_started":1}
I20260811 22:30:38.928968 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.921977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38195 (local address 127.0.0.1:56140)
0811 22:30:38.922381 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:38.922435 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:38.922610 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:38.923440 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:38.923447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:38.923961 (+ 514us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:38.924149 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.924156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.925717 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.925720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.927269 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.927280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.928308 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.928313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.928432 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.928480 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 22:30:38.928611 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 22:30:38.928801 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":91,"thread_start_us":103,"threads_started":1}
I20260811 22:30:38.929050 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.922999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.18.211.130:39645)
0811 22:30:38.923681 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.923696 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.923723 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:30:38.923799 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.923804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.924011 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.924201 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:38.924493 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.925500 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.927458 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.928139 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.928294 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.928445 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.928798 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.928897 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":159,"threads_started":1}
I20260811 22:30:38.929387 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.922803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:39645 (local address 127.0.0.1:49416)
0811 22:30:38.923095 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:38.923111 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:38.923211 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:38.924182 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:38.924185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:38.924206 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:38.924332 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.924339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.925641 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.925644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.927323 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.927333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.928508 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.928511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.928632 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.928652 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:30:38.928714 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 22:30:38.929258 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":552,"thread_start_us":125,"threads_started":1}
I20260811 22:30:38.929431 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.923190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:40505 (local address 127.0.0.1:41480)
0811 22:30:38.923539 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:38.923553 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:38.923641 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:38.924466 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:38.924468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:38.924488 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:38.924583 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.924590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.925951 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.925955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.927070 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:38.927082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.928517 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.928521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:38.928683 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.928709 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:30:38.928869 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:30:38.928941 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":45,"thread_start_us":115,"threads_started":1}
I20260811 22:30:38.929738 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:38.923604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41480 (local address 127.18.211.131:40505)
0811 22:30:38.923929 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:38.923943 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:38.923980 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:30:38.924068 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:38.924073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:38.924270 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:38.924418 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:38.924713 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.925819 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.927218 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:38.928184 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:38.928310 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:38.928445 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:38.929032 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 22:30:38.929575 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":150,"threads_started":1}
I20260811 22:30:38.932826 20505 tablet_service.cc:1467] Processing CreateTablet for tablet fa091563e0ee472a9cdb919deab6a9e8 (DEFAULT_TABLE table=test-table [id=3b57eb6bf8e548529637f6496c2bb575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:38.932845 20742 tablet_service.cc:1467] Processing CreateTablet for tablet fa091563e0ee472a9cdb919deab6a9e8 (DEFAULT_TABLE table=test-table [id=3b57eb6bf8e548529637f6496c2bb575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:38.932972 20982 tablet_service.cc:1467] Processing CreateTablet for tablet fa091563e0ee472a9cdb919deab6a9e8 (DEFAULT_TABLE table=test-table [id=3b57eb6bf8e548529637f6496c2bb575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:38.934101 20742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa091563e0ee472a9cdb919deab6a9e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:38.934103 20505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa091563e0ee472a9cdb919deab6a9e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:38.934129 20982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa091563e0ee472a9cdb919deab6a9e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:38.940838 21148 tablet_bootstrap.cc:492] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195: Bootstrap starting.
I20260811 22:30:38.942253 21150 tablet_bootstrap.cc:492] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa: Bootstrap starting.
I20260811 22:30:38.943284 21148 tablet_bootstrap.cc:654] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:38.943815 21152 tablet_bootstrap.cc:492] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93: Bootstrap starting.
I20260811 22:30:38.944329 21150 tablet_bootstrap.cc:654] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:38.944749 21148 log.cc:824] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:38.945290 21150 log.cc:824] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:38.946754 21152 tablet_bootstrap.cc:654] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:38.947247 21148 tablet_bootstrap.cc:492] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195: No bootstrap required, opened a new log
I20260811 22:30:38.947458 21148 ts_tablet_manager.cc:1397] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:30:38.947659 21152 log.cc:824] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:38.948627 21150 tablet_bootstrap.cc:492] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa: No bootstrap required, opened a new log
I20260811 22:30:38.948860 21150 ts_tablet_manager.cc:1397] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:30:38.949710 21152 tablet_bootstrap.cc:492] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93: No bootstrap required, opened a new log
I20260811 22:30:38.949900 21152 ts_tablet_manager.cc:1397] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:30:38.951576 21148 raft_consensus.cc:348] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:38.951905 21148 raft_consensus.cc:374] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:38.951970 21148 raft_consensus.cc:729] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c6dc306e4c461db9b93279d4e31195, State: Initialized, Role: FOLLOWER
I20260811 22:30:38.952453 21148 consensus_queue.cc:260] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:38.953267 21111 heartbeater.cc:503] Master 127.18.211.190:45081 was elected leader, sending a full tablet report...
I20260811 22:30:38.953339 21150 raft_consensus.cc:348] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:38.953588 21150 raft_consensus.cc:374] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:38.953620 21148 ts_tablet_manager.cc:1428] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260811 22:30:38.953635 21150 raft_consensus.cc:729] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b9a5df5a054ee1a4897278100fa5aa, State: Initialized, Role: FOLLOWER
I20260811 22:30:38.953867 21152 raft_consensus.cc:348] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:38.954172 21152 raft_consensus.cc:374] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:38.954099 21150 consensus_queue.cc:260] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:38.954239 21152 raft_consensus.cc:729] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b21b8bb75ac4a19992081eb29c34a93, State: Initialized, Role: FOLLOWER
I20260811 22:30:38.954734 21152 consensus_queue.cc:260] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:38.954874 20617 heartbeater.cc:503] Master 127.18.211.190:45081 was elected leader, sending a full tablet report...
I20260811 22:30:38.955160 21150 ts_tablet_manager.cc:1428] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:30:38.955657 21152 ts_tablet_manager.cc:1428] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 22:30:38.955982 20858 heartbeater.cc:503] Master 127.18.211.190:45081 was elected leader, sending a full tablet report...
I20260811 22:30:39.003850 21157 raft_consensus.cc:482] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:39.004098 21157 raft_consensus.cc:504] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:39.005409 21157 leader_election.cc:288] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b9a5df5a054ee1a4897278100fa5aa (127.18.211.129:38195), 4b21b8bb75ac4a19992081eb29c34a93 (127.18.211.130:39645)
I20260811 22:30:39.009948 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:39.005821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:35021 (local address 127.18.211.129:38195)
0811 22:30:39.005966 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:39.005972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:39.006006 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:30:39.006070 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:39.006072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:39.006139 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:39.006213 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:39.006614 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.007315 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.008695 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.009383 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.009484 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:39.009542 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:39.009778 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 22:30:39.009839 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:30:39.009953 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:39.005651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38195 (local address 127.18.211.131:35021)
0811 22:30:39.005808 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:39.005829 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:39.005949 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:39.006321 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:39.006324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:39.006343 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:39.006488 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:39.006495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.007462 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.007466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:39.008546 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:39.008556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.009507 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.009511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:39.009624 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:39.009651 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:30:39.009749 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 22:30:39.009818 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 22:30:39.010700 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:39.006111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:49117 (local address 127.18.211.130:39645)
0811 22:30:39.006243 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:39.006248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:39.006439 (+ 191us) server_negotiation.cc:407] Connection header received
0811 22:30:39.006529 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:39.006535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:39.006600 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:39.006671 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:39.007085 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.007832 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.009316 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.009962 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.010082 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:39.010135 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:39.010508 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 22:30:39.010582 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:30:39.010677 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:39.005943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:39645 (local address 127.18.211.131:49117)
0811 22:30:39.006329 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:39.006342 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:39.006438 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:39.006700 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:39.006703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:39.006720 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:39.006884 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:39.006891 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.007993 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.007996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:39.009113 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:39.009124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.010248 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.010252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:39.010367 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:39.010389 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:30:39.010483 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:30:39.010547 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":137,"threads_started":1}
I20260811 22:30:39.010798 20536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa091563e0ee472a9cdb919deab6a9e8" candidate_uuid: "31c6dc306e4c461db9b93279d4e31195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b9a5df5a054ee1a4897278100fa5aa" is_pre_election: true
I20260811 22:30:39.011093 20536 raft_consensus.cc:2454] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c6dc306e4c461db9b93279d4e31195 in term 0.
I20260811 22:30:39.011332 20776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa091563e0ee472a9cdb919deab6a9e8" candidate_uuid: "31c6dc306e4c461db9b93279d4e31195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b21b8bb75ac4a19992081eb29c34a93" is_pre_election: true
I20260811 22:30:39.011623 20776 raft_consensus.cc:2454] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c6dc306e4c461db9b93279d4e31195 in term 0.
I20260811 22:30:39.011652 20903 leader_election.cc:302] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c6dc306e4c461db9b93279d4e31195, 39b9a5df5a054ee1a4897278100fa5aa; no voters:
I20260811 22:30:39.011997 21157 raft_consensus.cc:2781] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:39.012107 21157 raft_consensus.cc:482] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:39.012156 21157 raft_consensus.cc:3037] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:39.013274 21157 raft_consensus.cc:504] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:39.013645 21157 leader_election.cc:288] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [CANDIDATE]: Term 1 election: Requested vote from peers 39b9a5df5a054ee1a4897278100fa5aa (127.18.211.129:38195), 4b21b8bb75ac4a19992081eb29c34a93 (127.18.211.130:39645)
I20260811 22:30:39.014025 20536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa091563e0ee472a9cdb919deab6a9e8" candidate_uuid: "31c6dc306e4c461db9b93279d4e31195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b9a5df5a054ee1a4897278100fa5aa"
I20260811 22:30:39.014056 20776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa091563e0ee472a9cdb919deab6a9e8" candidate_uuid: "31c6dc306e4c461db9b93279d4e31195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b21b8bb75ac4a19992081eb29c34a93"
I20260811 22:30:39.014204 20776 raft_consensus.cc:3037] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:39.014150 20536 raft_consensus.cc:3037] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:39.015321 20776 raft_consensus.cc:2454] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c6dc306e4c461db9b93279d4e31195 in term 1.
I20260811 22:30:39.015400 20536 raft_consensus.cc:2454] T fa091563e0ee472a9cdb919deab6a9e8 P 39b9a5df5a054ee1a4897278100fa5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c6dc306e4c461db9b93279d4e31195 in term 1.
I20260811 22:30:39.015669 20903 leader_election.cc:302] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c6dc306e4c461db9b93279d4e31195, 4b21b8bb75ac4a19992081eb29c34a93; no voters:
I20260811 22:30:39.015861 21157 raft_consensus.cc:2781] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:39.016130 21157 raft_consensus.cc:686] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [term 1 LEADER]: Becoming Leader. State: Replica: 31c6dc306e4c461db9b93279d4e31195, State: Running, Role: LEADER
I20260811 22:30:39.016489 21157 consensus_queue.cc:237] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } }
I20260811 22:30:39.019973 20287 catalog_manager.cc:5495] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31c6dc306e4c461db9b93279d4e31195 (127.18.211.131). New cstate: current_term: 1 leader_uuid: "31c6dc306e4c461db9b93279d4e31195" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b9a5df5a054ee1a4897278100fa5aa" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b21b8bb75ac4a19992081eb29c34a93" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31c6dc306e4c461db9b93279d4e31195" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40505 } health_report { overall_health: HEALTHY } } }
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[20206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:30:39.054863 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:39.044686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.18.211.129:38195)
0811 22:30:39.044860 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:39.044865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:39.045088 (+ 223us) server_negotiation.cc:407] Connection header received
0811 22:30:39.045307 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:39.045309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:39.045353 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:39.045442 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:39.046161 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.046669 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.047550 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.047760 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.047872 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:39.050690 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:39.050706 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:39.050725 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:39.050757 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:39.052776 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:39.053342 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:39.053349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:39.053350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:39.053445 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:39.053806 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:39.053809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:39.053811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:39.054092 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:39.054352 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:39.054547 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 22:30:39.054678 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[20206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[20206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:30:39.085491 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:39.076481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.18.211.129:38195)
0811 22:30:39.076688 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:39.076694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:39.076849 (+ 155us) server_negotiation.cc:407] Connection header received
0811 22:30:39.077040 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:39.077043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:39.077087 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:39.077178 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:39.077958 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.078488 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.079347 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.079581 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.079707 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:39.082346 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:39.082349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:39.082351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:39.082377 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:39.083765 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:39.084215 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:39.084218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:39.084219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:39.084296 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:39.084537 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:39.084539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:39.084540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:39.084714 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:39.084817 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:39.085143 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 22:30:39.085325 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[20206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[20206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
W20260811 22:30:39.113735 21116 tablet.cc:2369] T fa091563e0ee472a9cdb919deab6a9e8 P 31c6dc306e4c461db9b93279d4e31195: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:30:39.116245 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:39.106544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56174 (local address 127.18.211.129:38195)
0811 22:30:39.106702 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:39.106706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:39.106846 (+ 140us) server_negotiation.cc:407] Connection header received
0811 22:30:39.107034 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:39.107037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:39.107104 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:39.107192 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:39.108048 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.108614 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.109866 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:39.110094 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:39.110214 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:39.113058 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:39.113061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:39.113063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:39.113091 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:39.114641 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:39.115098 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:39.115101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:39.115102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:39.115196 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:39.115443 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:39.115445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:39.115446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:39.115613 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:39.115712 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:39.115924 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 22:30:39.116030 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 22:30:39.117738 20464 tablet_service.cc:2908] Rejecting scan request for tablet fa091563e0ee472a9cdb919deab6a9e8: Uninitialized: clean time has not yet been initialized
I20260811 22:30:39.119812 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20392
W20260811 22:30:39.127549 20865 tablet.cc:2369] T fa091563e0ee472a9cdb919deab6a9e8 P 4b21b8bb75ac4a19992081eb29c34a93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:30:39.141111 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20625
I20260811 22:30:39.156685 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20869
I20260811 22:30:39.176913 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20228
2026-08-11T22:30:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2737 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[21191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[21191](info): set up 2 sockets
Aug 11 22:30:39 dist-test-slave-8hhm krb5kdc[21191](info): commencing operation
krb5kdc: starting...
W20260811 22:30:40.483107 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:40 dist-test-slave-8hhm krb5kdc[21191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:40.620520 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43827
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:43827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:40.744501 21216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:40.744879 21216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:40.744961 21216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:40.749320 21216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:40.749410 21216 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:30:40.749441 21216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:40.749481 21216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:40.749516 21216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:40.749552 21216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:40.755915 21216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:43827
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:40.757151 21216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:40.758284 21216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:40.767104 21228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:40.767120 21229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:40.767115 21234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:40.767407 21216 server_base.cc:1034] running on GCE node
I20260811 22:30:40.768177 21216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:40.769433 21216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:40.770625 21216 hybrid_clock.cc:648] HybridClock initialized: now 1786487440770612 us; error 31 us; skew 500 ppm
Aug 11 22:30:40 dist-test-slave-8hhm krb5kdc[21191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:40.773576 21216 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:40.774520 21216 webserver.cc:460] Webserver started at http://127.18.211.190:46733/ using document root <none> and password file <none>
I20260811 22:30:40.775139 21216 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:40.775213 21216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:40.775441 21216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:40.777648 21216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "321ca9c11eaf44fe8989909ff7ecc7aa"
format_stamp: "Formatted at 2026-08-11 22:30:40 on dist-test-slave-8hhm"
server_key: "f3c2f26af9a78e662eda645a1586bbd4"
server_key_iv: "8127216faf3fc5f82cd5551e20ced86d"
server_key_version: "encryptionkey@0"
I20260811 22:30:40.778533 21216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "321ca9c11eaf44fe8989909ff7ecc7aa"
format_stamp: "Formatted at 2026-08-11 22:30:40 on dist-test-slave-8hhm"
server_key: "f3c2f26af9a78e662eda645a1586bbd4"
server_key_iv: "8127216faf3fc5f82cd5551e20ced86d"
server_key_version: "encryptionkey@0"
I20260811 22:30:40.782577 21216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:40.785116 21242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:40.786464 21216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:30:40.786626 21216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "321ca9c11eaf44fe8989909ff7ecc7aa"
format_stamp: "Formatted at 2026-08-11 22:30:40 on dist-test-slave-8hhm"
server_key: "f3c2f26af9a78e662eda645a1586bbd4"
server_key_iv: "8127216faf3fc5f82cd5551e20ced86d"
server_key_version: "encryptionkey@0"
I20260811 22:30:40.786746 21216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:40.802533 21216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:40.805644 21216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:40.805868 21216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:40.814828 21216 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:43827
I20260811 22:30:40.814837 21349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:43827 every 8 connection(s)
I20260811 22:30:40.815891 21216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:40.818637 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21216
I20260811 22:30:40.818732 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:40.819022 19278 external_mini_cluster.cc:1428] Setting key d9e8d840d38da44c04f04e703fac91fe
I20260811 22:30:40.819121 21350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:40.826961 21350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa: Bootstrap starting.
Aug 11 22:30:40 dist-test-slave-8hhm krb5kdc[21191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:40.829660 21350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:40.830607 21350 log.cc:824] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:40.832898 21350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa: No bootstrap required, opened a new log
I20260811 22:30:40.835580 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:40.820562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.18.211.190:43827)
0811 22:30:40.821028 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:40.821048 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:40.821092 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:30:40.822146 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:40.822173 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:40.822632 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:40.823056 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:40.823788 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:40.825377 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:40.826256 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:40.826964 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:40.827144 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:40.829879 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:40.829885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:40.829900 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:40.829932 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:40.832585 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:40.833066 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:40.833077 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:40.833082 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:40.833281 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:40.833588 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:40.833592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:40.833594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:40.834014 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:40.834230 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:40.834539 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:30:40.834843 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":120,"threads_started":1}
I20260811 22:30:40.836333 21350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43827 } }
I20260811 22:30:40.836604 21350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:40.836697 21350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 321ca9c11eaf44fe8989909ff7ecc7aa, State: Initialized, Role: FOLLOWER
I20260811 22:30:40.837333 21350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43827 } }
I20260811 22:30:40.837538 21359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:40.837682 21359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43827 } }
I20260811 22:30:40.837515 21350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:40.838122 21350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:40.838104 21359 leader_election.cc:302] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 321ca9c11eaf44fe8989909ff7ecc7aa; no voters:
I20260811 22:30:40.838232 21350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:40.838515 21359 leader_election.cc:288] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260811 22:30:40.839329 21350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43827 } }
I20260811 22:30:40.839483 21350 leader_election.cc:302] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 321ca9c11eaf44fe8989909ff7ecc7aa; no voters:
I20260811 22:30:40.839530 21363 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 22:30:40.839607 21350 leader_election.cc:288] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:40.839879 21359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:40.840095 21359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [term 1 LEADER]: Becoming Leader. State: Replica: 321ca9c11eaf44fe8989909ff7ecc7aa, State: Running, Role: LEADER
I20260811 22:30:40.840471 21359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43827 } }
I20260811 22:30:40.841046 21350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:40.842002 21363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 321ca9c11eaf44fe8989909ff7ecc7aa. Latest consensus state: current_term: 1 leader_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43827 } } }
I20260811 22:30:40.842254 21363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:40.842648 21369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321ca9c11eaf44fe8989909ff7ecc7aa" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43827 } } }
I20260811 22:30:40.842705 21380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:40.842828 21369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:40.845930 21380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:40.852381 21380 catalog_manager.cc:1347] Generated new cluster ID: ef4108a173024e88baaf0557c2f08d1d
I20260811 22:30:40.852473 21380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:40.860452 21380 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:40.861500 21380 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:40.868491 21380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa: Generated new TSK 0
I20260811 22:30:40.869407 21380 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:30:40.908912 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43827
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:41.027441 21404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:41.027765 21404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:41.027844 21404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:41.031890 21404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:41.032001 21404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:41.032073 21404 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:30:41.032156 21404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:30:41.037449 21404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:43827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:41.038571 21404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:41.039657 21404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:41.049639 21417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:41.049643 21420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:41.049702 21404 server_base.cc:1034] running on GCE node
W20260811 22:30:41.049643 21416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:41.050300 21404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:41.051018 21404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:41.052194 21404 hybrid_clock.cc:648] HybridClock initialized: now 1786487441052176 us; error 29 us; skew 500 ppm
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:41.055222 21404 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:30:41.056157 21404 webserver.cc:460] Webserver started at http://127.18.211.129:42313/ using document root <none> and password file <none>
I20260811 22:30:41.056771 21404 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:41.056872 21404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:41.057125 21404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:41.059551 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "a3be78d4aef047bca9a2f721f2278021"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "016c251d133dfbc34519692cb5020e65"
server_key_iv: "9935852e0f7d70dc6557785ba9baa566"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.060190 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "a3be78d4aef047bca9a2f721f2278021"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "016c251d133dfbc34519692cb5020e65"
server_key_iv: "9935852e0f7d70dc6557785ba9baa566"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.064471 21404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:30:41.067152 21436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:41.068418 21404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:30:41.068569 21404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "a3be78d4aef047bca9a2f721f2278021"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "016c251d133dfbc34519692cb5020e65"
server_key_iv: "9935852e0f7d70dc6557785ba9baa566"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.068703 21404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:41.089553 21404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:41.092954 21404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:41.093176 21404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:41.094835 21404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:41.094913 21404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:41.094982 21404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:41.095018 21404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:41.106412 21404 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:41669
I20260811 22:30:41.106496 21642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:41669 every 8 connection(s)
I20260811 22:30:41.107366 21404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:30:41.107587 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21404
I20260811 22:30:41.107699 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:30:41.108008 19278 external_mini_cluster.cc:1428] Setting key 2b460f373917d1e96f3343069f28244f
I20260811 22:30:41.110330 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.095780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:41881 (local address 127.18.211.190:43827)
0811 22:30:41.095908 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.095912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.097061 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 22:30:41.098412 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.098417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.098487 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.098593 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.100169 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.101097 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.102564 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.102824 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.102956 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.105802 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.105806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.105809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.105844 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.107797 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.108562 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.108566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.108568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.108680 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.109131 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.109134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.109136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.109396 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.109499 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.110009 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.110162 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:30:41.111212 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.096363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43827 (local address 127.18.211.129:41881)
0811 22:30:41.096904 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.096950 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.098135 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.098805 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.098818 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.099245 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:41.099966 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.099985 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.101303 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.101307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.102423 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.102434 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.103012 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.103016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.103191 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.103813 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:41.103830 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:41.105611 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:41.107938 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.107944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.107958 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.108312 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.108853 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.108868 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.108869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.109006 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.109496 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:41.109502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:41.109762 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.109952 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.110277 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":154,"threads_started":1}
I20260811 22:30:41.112663 21643 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43827
I20260811 22:30:41.112911 21643 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:41.113511 21643 heartbeater.cc:511] Master 127.18.211.190:43827 requested a full tablet report, sending...
I20260811 22:30:41.115079 21268 ts_manager.cc:194] Registered new tserver with Master: a3be78d4aef047bca9a2f721f2278021 (127.18.211.129:41669)
I20260811 22:30:41.116782 21268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:41881
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:30:41.123430 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.115490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43827 (local address 127.18.211.129:52455)
0811 22:30:41.115797 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.115814 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.115929 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.116216 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.116219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.116457 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:41.116583 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.116590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.117791 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.117796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.118382 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.118391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.118882 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.118885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.118985 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.119559 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:41.119572 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:41.119928 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:41.121839 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.121842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.121844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.122056 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.122368 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.122370 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.122373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.122459 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.122988 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:41.122990 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:41.123073 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.123159 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.123274 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":108,"threads_started":1}
I20260811 22:30:41.123564 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.115331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:52455 (local address 127.18.211.190:43827)
0811 22:30:41.115477 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.115482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.115911 (+ 429us) server_negotiation.cc:407] Connection header received
0811 22:30:41.116029 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.116034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.116093 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.116184 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.116718 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.117634 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.118521 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.118758 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.118876 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.120092 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.120096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.120099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.120124 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.121713 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.122166 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.122171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.122173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.122263 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.122604 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.122609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.122611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.122883 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.122984 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.123244 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.123359 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":63}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:30:41.144107 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43827
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:41.263477 21656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:41.263813 21656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:41.263906 21656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:41.268388 21656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:41.268496 21656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:41.268567 21656 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:30:41.268656 21656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:30:41.274379 21656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:43827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:41.275540 21656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:41.276612 21656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:41.286604 21665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:41.286604 21663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:41.286769 21656 server_base.cc:1034] running on GCE node
W20260811 22:30:41.286644 21662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:41.287395 21656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:41.288045 21656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:41.289326 21656 hybrid_clock.cc:648] HybridClock initialized: now 1786487441289285 us; error 50 us; skew 500 ppm
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:41.292549 21656 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:30:41.293588 21656 webserver.cc:460] Webserver started at http://127.18.211.130:44771/ using document root <none> and password file <none>
I20260811 22:30:41.294138 21656 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:41.294199 21656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:41.294371 21656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:41.296505 21656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "37260d27d1314235aeeefb62fb097f0e"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "f4af07e9667362bbfd8f44b164f4603b"
server_key_iv: "f147afc68a2ba03233c8adac0c7666dd"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.297092 21656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "37260d27d1314235aeeefb62fb097f0e"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "f4af07e9667362bbfd8f44b164f4603b"
server_key_iv: "f147afc68a2ba03233c8adac0c7666dd"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.301029 21656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:30:41.303673 21675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:41.305003 21656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:30:41.305166 21656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "37260d27d1314235aeeefb62fb097f0e"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "f4af07e9667362bbfd8f44b164f4603b"
server_key_iv: "f147afc68a2ba03233c8adac0c7666dd"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.305346 21656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:41.331003 21656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:41.334648 21656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:41.335179 21656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:41.336797 21656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:41.336850 21656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:41.336894 21656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:41.336910 21656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:41.348449 21656 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:45697
I20260811 22:30:41.348524 21879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:45697 every 8 connection(s)
I20260811 22:30:41.349677 21656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:30:41.352291 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.337749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:35267 (local address 127.18.211.190:43827)
0811 22:30:41.337885 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.337888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.338991 (+ 1103us) server_negotiation.cc:407] Connection header received
0811 22:30:41.340118 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.340122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.340218 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.340313 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.342150 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.342920 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.344347 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.344587 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.344719 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.348312 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.348316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.348319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.348354 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.349960 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.350576 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.350579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.350581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.350684 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.351094 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.351098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.351099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.351333 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.351428 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.351997 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.352115 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:30:41.352535 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21656
I20260811 22:30:41.352613 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:30:41.352839 19278 external_mini_cluster.cc:1428] Setting key de852dc34c594891d7a56e9b4ede4a11
I20260811 22:30:41.353122 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.338297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43827 (local address 127.18.211.130:35267)
0811 22:30:41.338824 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.338869 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.339827 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.340529 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.340541 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.341127 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:41.341945 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.341971 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.343138 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.343144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.344189 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.344202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.344784 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.344789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.344974 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.346250 (+ 1276us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:41.346274 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:41.348056 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:41.350090 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.350096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.350116 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.350465 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.350795 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.350797 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.350798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.350936 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.351475 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:41.351481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:41.351756 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.351935 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.352210 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":154,"threads_started":1}
I20260811 22:30:41.354558 21880 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43827
I20260811 22:30:41.354769 21880 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:41.355317 21880 heartbeater.cc:511] Master 127.18.211.190:43827 requested a full tablet report, sending...
I20260811 22:30:41.356501 21268 ts_manager.cc:194] Registered new tserver with Master: 37260d27d1314235aeeefb62fb097f0e (127.18.211.130:45697)
I20260811 22:30:41.357409 21268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:35267
I20260811 22:30:41.365422 21889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.357157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43827 (local address 127.18.211.130:57875)
0811 22:30:41.357512 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.357548 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.357663 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.357990 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.357992 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.358187 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:41.358282 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.358287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.359256 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.359258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.359807 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.359815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.360443 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.360446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.360512 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.361105 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:41.361120 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:41.361535 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:41.363606 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.363609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.363611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.363826 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.364153 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.364156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.364157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.364273 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.364870 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:41.364872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:41.364960 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.365049 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.365171 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":126,"threads_started":1}
I20260811 22:30:41.365549 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.357237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:57875 (local address 127.18.211.190:43827)
0811 22:30:41.357361 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.357366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.357609 (+ 243us) server_negotiation.cc:407] Connection header received
0811 22:30:41.357780 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.357783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.357827 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.357876 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.358410 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.359120 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.359933 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.360323 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.360449 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.361687 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.361694 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.361701 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.361734 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.363481 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.363938 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.363942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.363945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.364049 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.364418 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.364422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.364424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.364762 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.364872 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.365153 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.365350 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:30:41.388437 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43827
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:41.508693 21892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:41.508980 21892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:41.509076 21892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:41.513356 21892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:41.513427 21892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:41.513502 21892 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:30:41.513597 21892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:30:41.519039 21892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:43827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:41.520179 21892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:41.521344 21892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:41.532737 21909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:41.532752 21905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:41.532930 21892 server_base.cc:1034] running on GCE node
W20260811 22:30:41.532752 21903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:41.533560 21892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:41.534248 21892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:41.535409 21892 hybrid_clock.cc:648] HybridClock initialized: now 1786487441535376 us; error 48 us; skew 500 ppm
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:41.538300 21892 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:30:41.539222 21892 webserver.cc:460] Webserver started at http://127.18.211.131:34369/ using document root <none> and password file <none>
I20260811 22:30:41.539795 21892 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:41.539860 21892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:41.540107 21892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:41.542342 21892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "bfb1ab26c14645918163bc48384cc67c"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "ae7a263644854fd1dda3edd714886bf2"
server_key_iv: "53a9d2dcf23a6675ed68c50d32fbc0fc"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.542997 21892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "bfb1ab26c14645918163bc48384cc67c"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "ae7a263644854fd1dda3edd714886bf2"
server_key_iv: "53a9d2dcf23a6675ed68c50d32fbc0fc"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.546919 21892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:41.549669 21919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:41.550922 21892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:41.551024 21892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "bfb1ab26c14645918163bc48384cc67c"
format_stamp: "Formatted at 2026-08-11 22:30:41 on dist-test-slave-8hhm"
server_key: "ae7a263644854fd1dda3edd714886bf2"
server_key_iv: "53a9d2dcf23a6675ed68c50d32fbc0fc"
server_key_version: "encryptionkey@0"
I20260811 22:30:41.551156 21892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:41.573434 21892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:41.576735 21892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:41.576953 21892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:41.578588 21892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:41.578644 21892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:41.578692 21892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:41.578744 21892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:41.590234 21892 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:44151
I20260811 22:30:41.590325 22120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:44151 every 8 connection(s)
I20260811 22:30:41.591270 21892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:30:41.592767 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.579459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:40513 (local address 127.18.211.190:43827)
0811 22:30:41.579647 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.579654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.580533 (+ 879us) server_negotiation.cc:407] Connection header received
0811 22:30:41.581676 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.581679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.581752 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.581871 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.583654 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.584265 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.585537 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.585770 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.585894 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.588600 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.588604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.588611 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.588637 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.590438 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.590989 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.590993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.590995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.591101 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.591496 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.591499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.591501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.591785 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.591888 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.592532 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.592645 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:30:41.593705 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.579860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43827 (local address 127.18.211.131:40513)
0811 22:30:41.580355 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.580407 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.581433 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.582118 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.582131 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.582686 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:41.583455 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.583484 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.584403 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.584407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.585401 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.585411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.585950 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.585954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.586126 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.586842 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:41.586858 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:41.588406 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:41.590546 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.590552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.590568 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.590874 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.591219 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.591222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.591224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.591399 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.591941 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:41.591947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:41.592264 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.592461 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.592725 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":154,"threads_started":1}
I20260811 22:30:41.594942 22122 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43827
I20260811 22:30:41.595094 22122 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:41.595702 22122 heartbeater.cc:511] Master 127.18.211.190:43827 requested a full tablet report, sending...
I20260811 22:30:41.596614 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21892
I20260811 22:30:41.596740 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:30:41.596899 21267 ts_manager.cc:194] Registered new tserver with Master: bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151)
I20260811 22:30:41.597043 19278 external_mini_cluster.cc:1428] Setting key 84500c1c6eaf65fbf789c7fd3ea241d8
I20260811 22:30:41.597702 21267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:40513
I20260811 22:30:41.599439 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:30:41.605983 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.597531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:52537 (local address 127.18.211.190:43827)
0811 22:30:41.597636 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.597641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.598112 (+ 471us) server_negotiation.cc:407] Connection header received
0811 22:30:41.598302 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.598306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.598357 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.598453 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.598993 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.599729 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.600934 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.601142 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.601284 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.602556 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.602562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.602567 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.602597 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.604152 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.604664 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.604666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.604668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.604752 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.605067 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.605069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.605070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.605299 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.605380 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.605754 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.605865 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:30:41.606109 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.597731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43827 (local address 127.18.211.131:52537)
0811 22:30:41.598028 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.598046 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.598184 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.598471 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.598474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.598712 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:41.598851 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.598859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.599875 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.599878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.600813 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.600829 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.601286 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.601292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.601423 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.601974 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:41.601996 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:41.602407 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:41.604274 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.604280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.604285 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.604562 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.604845 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.604851 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.604856 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.604965 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.605403 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:41.605410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:41.605533 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.605665 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.605845 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":113,"threads_started":1}
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:41.630219 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.621332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.18.211.190:43827)
0811 22:30:41.621467 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.621471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.621484 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:30:41.621527 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.621529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.621568 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.621668 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.622434 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.622945 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.623804 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.624028 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.624139 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.626695 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.626699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.626702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.626731 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.628211 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.628715 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.628720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.628722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.628817 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.629144 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.629149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.629151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.629421 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.629580 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.629867 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.630021 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:30:41.633767 21267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:30:41.636828 21267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:41.652273 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.646352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:44151 (local address 127.0.0.1:56858)
0811 22:30:41.646635 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.646685 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.646829 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.647992 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.647998 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.648022 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:41.648204 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.648211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.649503 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.649506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.650761 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.650773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.651647 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.651650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.651800 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.651856 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.651971 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.652108 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"spinlock_wait_cycles":11392,"thread_start_us":111,"threads_started":1}
I20260811 22:30:41.652299 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.647080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.18.211.131:44151)
0811 22:30:41.647433 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.647448 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.647472 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:30:41.647551 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.647555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.647793 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.647998 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:41.648339 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.649364 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.650869 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.651523 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.651677 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.651831 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.652029 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.652130 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":110,"threads_started":1}
I20260811 22:30:41.652747 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.645938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41669 (local address 127.0.0.1:38316)
0811 22:30:41.646625 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.646681 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.646830 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.648454 (+ 1624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.648457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.648470 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:41.648657 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.648665 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.650047 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.650051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.651244 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.651255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.652371 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.652376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.652484 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.652509 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.652568 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.652619 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":56,"spinlock_wait_cycles":3584,"thread_start_us":125,"threads_started":1}
I20260811 22:30:41.653503 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.646145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.18.211.129:41669)
0811 22:30:41.647998 (+ 1853us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.648005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.648028 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:30:41.648102 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.648107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.648317 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.648600 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:41.648749 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.649916 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.651392 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.652218 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.653081 (+ 863us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.653239 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.653289 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.653378 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1746,"thread_start_us":85,"threads_started":1}
I20260811 22:30:41.653543 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.646767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:45697 (local address 127.0.0.1:58600)
0811 22:30:41.647103 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.647116 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.647208 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.648377 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.648383 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.648405 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:41.648565 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.648573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.650047 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.650051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.651689 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.651709 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.653095 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.653098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.653172 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.653189 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.653321 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.653390 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":42,"thread_start_us":103,"threads_started":1}
I20260811 22:30:41.654103 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.647474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.18.211.130:45697)
0811 22:30:41.647947 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.647963 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.647984 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:30:41.648059 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.648064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.648239 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.648408 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:41.648701 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.649918 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.651849 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.652756 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.652903 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.653032 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.653842 (+ 810us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.653949 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":126,"threads_started":1}
I20260811 22:30:41.656200 22007 tablet_service.cc:1467] Processing CreateTablet for tablet e48d61b99e394d3abdda20fcd8e0fedd (DEFAULT_TABLE table=test-table [id=4b8451fe8905478183130cdc11e7b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:41.656898 21761 tablet_service.cc:1467] Processing CreateTablet for tablet e48d61b99e394d3abdda20fcd8e0fedd (DEFAULT_TABLE table=test-table [id=4b8451fe8905478183130cdc11e7b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:41.656901 21518 tablet_service.cc:1467] Processing CreateTablet for tablet e48d61b99e394d3abdda20fcd8e0fedd (DEFAULT_TABLE table=test-table [id=4b8451fe8905478183130cdc11e7b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:41.657475 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48d61b99e394d3abdda20fcd8e0fedd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.657923 21518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48d61b99e394d3abdda20fcd8e0fedd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.657948 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48d61b99e394d3abdda20fcd8e0fedd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.664472 22159 tablet_bootstrap.cc:492] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Bootstrap starting.
I20260811 22:30:41.665688 22161 tablet_bootstrap.cc:492] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Bootstrap starting.
I20260811 22:30:41.666056 22160 tablet_bootstrap.cc:492] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Bootstrap starting.
I20260811 22:30:41.667212 22159 tablet_bootstrap.cc:654] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.667649 22161 tablet_bootstrap.cc:654] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.668087 22160 tablet_bootstrap.cc:654] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.668160 22159 log.cc:824] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:41.668612 22161 log.cc:824] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:41.669044 22160 log.cc:824] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:41.670531 22159 tablet_bootstrap.cc:492] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: No bootstrap required, opened a new log
I20260811 22:30:41.670751 22159 ts_tablet_manager.cc:1397] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:30:41.671043 22160 tablet_bootstrap.cc:492] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: No bootstrap required, opened a new log
I20260811 22:30:41.671223 22160 ts_tablet_manager.cc:1397] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:30:41.675045 22161 tablet_bootstrap.cc:492] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: No bootstrap required, opened a new log
I20260811 22:30:41.675263 22161 ts_tablet_manager.cc:1397] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 22:30:41.675186 22159 raft_consensus.cc:348] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.675477 22159 raft_consensus.cc:374] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.675530 22159 raft_consensus.cc:729] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37260d27d1314235aeeefb62fb097f0e, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.675897 22160 raft_consensus.cc:348] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.676005 22159 consensus_queue.cc:260] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.676139 22160 raft_consensus.cc:374] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.676195 22160 raft_consensus.cc:729] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3be78d4aef047bca9a2f721f2278021, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.676631 22160 consensus_queue.cc:260] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.676844 21880 heartbeater.cc:503] Master 127.18.211.190:43827 was elected leader, sending a full tablet report...
I20260811 22:30:41.677117 22159 ts_tablet_manager.cc:1428] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:30:41.677558 22160 ts_tablet_manager.cc:1428] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 22:30:41.677942 21643 heartbeater.cc:503] Master 127.18.211.190:43827 was elected leader, sending a full tablet report...
I20260811 22:30:41.679433 22161 raft_consensus.cc:348] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.679832 22161 raft_consensus.cc:374] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.679922 22161 raft_consensus.cc:729] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb1ab26c14645918163bc48384cc67c, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.680390 22161 consensus_queue.cc:260] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.681177 22122 heartbeater.cc:503] Master 127.18.211.190:43827 was elected leader, sending a full tablet report...
I20260811 22:30:41.681496 22161 ts_tablet_manager.cc:1428] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:30:41.741274 22167 raft_consensus.cc:482] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:41.741539 22167 raft_consensus.cc:504] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.742982 22167 leader_election.cc:288] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3be78d4aef047bca9a2f721f2278021 (127.18.211.129:41669), bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151)
I20260811 22:30:41.747141 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.743216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41669 (local address 127.18.211.130:40643)
0811 22:30:41.743391 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.743408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.743506 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.743931 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.743934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.743953 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:41.744098 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.744107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.744899 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.744905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.745811 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.745821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.746719 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.746723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.746829 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.746855 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.746933 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.746999 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 22:30:41.747151 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.743439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:40643 (local address 127.18.211.129:41669)
0811 22:30:41.743670 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.743676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.743693 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:30:41.743756 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.743759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.743815 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.743893 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:41.744232 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.744774 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.745944 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.746594 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.746701 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.746743 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.746958 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.747053 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260811 22:30:41.747766 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.743256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:44151 (local address 127.18.211.130:48103)
0811 22:30:41.743655 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.743669 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.743775 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.744067 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.744069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.744084 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:41.744214 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.744220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.744956 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.744958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.746440 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.746451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.747469 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.747472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.747547 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.747563 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.747623 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.747668 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":169,"threads_started":1}
I20260811 22:30:41.747870 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.743387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:48103 (local address 127.18.211.131:44151)
0811 22:30:41.743603 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.743609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.743755 (+ 146us) server_negotiation.cc:407] Connection header received
0811 22:30:41.743894 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.743900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.743952 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.744033 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:41.744328 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.744808 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.746598 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.747169 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.747312 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.747373 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.747677 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.747752 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 22:30:41.747845 21556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48d61b99e394d3abdda20fcd8e0fedd" candidate_uuid: "37260d27d1314235aeeefb62fb097f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3be78d4aef047bca9a2f721f2278021" is_pre_election: true
I20260811 22:30:41.748190 21556 raft_consensus.cc:2454] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37260d27d1314235aeeefb62fb097f0e in term 0.
I20260811 22:30:41.748616 22039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48d61b99e394d3abdda20fcd8e0fedd" candidate_uuid: "37260d27d1314235aeeefb62fb097f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1ab26c14645918163bc48384cc67c" is_pre_election: true
I20260811 22:30:41.748744 21679 leader_election.cc:302] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37260d27d1314235aeeefb62fb097f0e, a3be78d4aef047bca9a2f721f2278021; no voters:
I20260811 22:30:41.748914 22039 raft_consensus.cc:2454] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37260d27d1314235aeeefb62fb097f0e in term 0.
I20260811 22:30:41.749043 22167 raft_consensus.cc:2781] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:41.749130 22167 raft_consensus.cc:482] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:41.749243 22167 raft_consensus.cc:3037] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:41.750316 22167 raft_consensus.cc:504] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.750710 22167 leader_election.cc:288] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [CANDIDATE]: Term 1 election: Requested vote from peers a3be78d4aef047bca9a2f721f2278021 (127.18.211.129:41669), bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151)
I20260811 22:30:41.751096 21556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48d61b99e394d3abdda20fcd8e0fedd" candidate_uuid: "37260d27d1314235aeeefb62fb097f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3be78d4aef047bca9a2f721f2278021"
I20260811 22:30:41.751165 22039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48d61b99e394d3abdda20fcd8e0fedd" candidate_uuid: "37260d27d1314235aeeefb62fb097f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1ab26c14645918163bc48384cc67c"
I20260811 22:30:41.751251 21556 raft_consensus.cc:3037] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:41.751308 22039 raft_consensus.cc:3037] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:41.752336 21556 raft_consensus.cc:2454] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37260d27d1314235aeeefb62fb097f0e in term 1.
I20260811 22:30:41.752705 21679 leader_election.cc:302] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37260d27d1314235aeeefb62fb097f0e, a3be78d4aef047bca9a2f721f2278021; no voters:
I20260811 22:30:41.752815 22039 raft_consensus.cc:2454] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37260d27d1314235aeeefb62fb097f0e in term 1.
I20260811 22:30:41.752907 22167 raft_consensus.cc:2781] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:41.753152 22167 raft_consensus.cc:686] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 1 LEADER]: Becoming Leader. State: Replica: 37260d27d1314235aeeefb62fb097f0e, State: Running, Role: LEADER
I20260811 22:30:41.753515 22167 consensus_queue.cc:237] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.757402 21267 catalog_manager.cc:5495] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 37260d27d1314235aeeefb62fb097f0e (127.18.211.130). New cstate: current_term: 1 leader_uuid: "37260d27d1314235aeeefb62fb097f0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } health_report { overall_health: UNKNOWN } } }
I20260811 22:30:41.779907 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.776217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.18.211.130:45697)
0811 22:30:41.776348 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.776351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.776365 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:30:41.776529 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.776531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.776587 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.776674 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:41.776911 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.777541 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.778406 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.778626 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.778723 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.779095 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:41.779561 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:41.779665 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.779737 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.779793 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:30:41.785916 22039 raft_consensus.cc:1261] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Refusing update from remote peer 37260d27d1314235aeeefb62fb097f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:41.785916 21556 raft_consensus.cc:1261] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Refusing update from remote peer 37260d27d1314235aeeefb62fb097f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:41.786697 22167 consensus_queue.cc:1035] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:41.786909 22173 consensus_queue.cc:1035] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:41.806176 21268 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:41.806457 21268 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:41.809268 21268 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa: Sending DeleteTablet for 3 replicas of tablet e48d61b99e394d3abdda20fcd8e0fedd
I20260811 22:30:41.810295 22007 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d61b99e394d3abdda20fcd8e0fedd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:41 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56858
I20260811 22:30:41.810333 21518 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d61b99e394d3abdda20fcd8e0fedd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:41 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:38316
I20260811 22:30:41.811020 22191 tablet_replica.cc:330] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: stopping tablet replica
I20260811 22:30:41.811031 21761 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d61b99e394d3abdda20fcd8e0fedd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:41 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:58600
I20260811 22:30:41.811049 21267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:30:41.811491 22192 tablet_replica.cc:330] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: stopping tablet replica
W20260811 22:30:41.811496 21267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:41.811725 22191 raft_consensus.cc:2229] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:41.811818 22192 raft_consensus.cc:2229] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:41.812166 22192 raft_consensus.cc:2258] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:41.812168 22191 raft_consensus.cc:2258] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:41.812898 22195 tablet_replica.cc:330] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: stopping tablet replica
I20260811 22:30:41.813256 22195 raft_consensus.cc:2229] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:30:41.813637 22195 raft_consensus.cc:2258] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:41.815954 22195 ts_tablet_manager.cc:1905] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:41.814517 22191 ts_tablet_manager.cc:1905] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:41.817262 22192 ts_tablet_manager.cc:1905] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:41.819844 21518 tablet_service.cc:1467] Processing CreateTablet for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea (DEFAULT_TABLE table=test-table [id=3343329fe87e48a59641ea58d44b1540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:41.820139 21518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.820376 22192 ts_tablet_manager.cc:1918] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:30:41.820494 22192 log.cc:1196] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/e48d61b99e394d3abdda20fcd8e0fedd
I20260811 22:30:41.820868 22192 ts_tablet_manager.cc:1939] T e48d61b99e394d3abdda20fcd8e0fedd P a3be78d4aef047bca9a2f721f2278021: Deleting consensus metadata
I20260811 22:30:41.822072 22195 ts_tablet_manager.cc:1918] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:30:41.822312 22160 tablet_bootstrap.cc:492] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Bootstrap starting.
I20260811 22:30:41.822367 21244 catalog_manager.cc:4841] TS a3be78d4aef047bca9a2f721f2278021 (127.18.211.129:41669): tablet e48d61b99e394d3abdda20fcd8e0fedd (table test-table [id=4b8451fe8905478183130cdc11e7b922]) successfully deleted
I20260811 22:30:41.822937 22007 tablet_service.cc:1467] Processing CreateTablet for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea (DEFAULT_TABLE table=test-table [id=3343329fe87e48a59641ea58d44b1540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:41.822570 21761 tablet_service.cc:1467] Processing CreateTablet for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea (DEFAULT_TABLE table=test-table [id=3343329fe87e48a59641ea58d44b1540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:41.823274 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.823293 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.823345 22160 tablet_bootstrap.cc:654] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.824707 22160 tablet_bootstrap.cc:492] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: No bootstrap required, opened a new log
I20260811 22:30:41.824795 22160 ts_tablet_manager.cc:1397] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:30:41.825328 22160 raft_consensus.cc:348] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.825444 22160 raft_consensus.cc:374] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.825476 22160 raft_consensus.cc:729] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3be78d4aef047bca9a2f721f2278021, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.822988 22191 ts_tablet_manager.cc:1918] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:30:41.825693 22159 tablet_bootstrap.cc:492] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Bootstrap starting.
I20260811 22:30:41.825707 22191 log.cc:1196] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/e48d61b99e394d3abdda20fcd8e0fedd
I20260811 22:30:41.825805 22160 consensus_queue.cc:260] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.826152 22191 ts_tablet_manager.cc:1939] T e48d61b99e394d3abdda20fcd8e0fedd P bfb1ab26c14645918163bc48384cc67c: Deleting consensus metadata
I20260811 22:30:41.826162 22160 ts_tablet_manager.cc:1428] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:41.826759 22159 tablet_bootstrap.cc:654] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.827538 22195 log.cc:1196] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/e48d61b99e394d3abdda20fcd8e0fedd
I20260811 22:30:41.827925 22195 ts_tablet_manager.cc:1939] T e48d61b99e394d3abdda20fcd8e0fedd P 37260d27d1314235aeeefb62fb097f0e: Deleting consensus metadata
I20260811 22:30:41.828296 21243 catalog_manager.cc:4841] TS bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151): tablet e48d61b99e394d3abdda20fcd8e0fedd (table test-table [id=4b8451fe8905478183130cdc11e7b922]) successfully deleted
I20260811 22:30:41.829113 21244 catalog_manager.cc:4841] TS 37260d27d1314235aeeefb62fb097f0e (127.18.211.130:45697): tablet e48d61b99e394d3abdda20fcd8e0fedd (table test-table [id=4b8451fe8905478183130cdc11e7b922]) successfully deleted
I20260811 22:30:41.829353 22161 tablet_bootstrap.cc:492] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Bootstrap starting.
I20260811 22:30:41.830379 22161 tablet_bootstrap.cc:654] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.831667 22161 tablet_bootstrap.cc:492] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: No bootstrap required, opened a new log
I20260811 22:30:41.831804 22161 ts_tablet_manager.cc:1397] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:30:41.832258 22161 raft_consensus.cc:348] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.832387 22161 raft_consensus.cc:374] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.832432 22161 raft_consensus.cc:729] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb1ab26c14645918163bc48384cc67c, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.832578 22161 consensus_queue.cc:260] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.833567 22161 ts_tablet_manager.cc:1428] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:30:41.838075 22159 tablet_bootstrap.cc:492] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: No bootstrap required, opened a new log
I20260811 22:30:41.838223 22159 ts_tablet_manager.cc:1397] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260811 22:30:41.838685 22159 raft_consensus.cc:348] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.838812 22159 raft_consensus.cc:374] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.838853 22159 raft_consensus.cc:729] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37260d27d1314235aeeefb62fb097f0e, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.838995 22159 consensus_queue.cc:260] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.839282 22159 ts_tablet_manager.cc:1428] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260811 22:30:41.842317 22123 tablet.cc:2369] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:30:41.851076 21881 tablet.cc:2369] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:30:41.859540 21644 tablet.cc:2369] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:30:41.889333 22168 raft_consensus.cc:482] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:41.889503 22168 raft_consensus.cc:504] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.890903 22168 leader_election.cc:288] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37260d27d1314235aeeefb62fb097f0e (127.18.211.130:45697), bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151)
I20260811 22:30:41.895977 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.891136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:44151 (local address 127.18.211.129:44491)
0811 22:30:41.891669 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.891684 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.891814 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.892077 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.892080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.892100 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:41.892271 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.892279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.893094 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.893098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.894556 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.894569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.895447 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.895450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.895580 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.895602 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.895696 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.895773 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":70,"thread_start_us":87,"threads_started":1}
I20260811 22:30:41.895953 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.891245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:44491 (local address 127.18.211.131:44151)
0811 22:30:41.891374 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.891378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.891783 (+ 405us) server_negotiation.cc:407] Connection header received
0811 22:30:41.891915 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.891921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.891968 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.892045 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:41.892425 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.892965 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.894702 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.895312 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.895463 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.895520 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.895728 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.895813 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:30:41.896781 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.891544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:44927 (local address 127.18.211.130:45697)
0811 22:30:41.891666 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.891715 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.891732 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:30:41.891790 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.891793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.891857 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.891929 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:41.892440 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.893204 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.895611 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.896197 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.896316 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.896362 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.896567 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.896648 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:30:41.896891 22039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e0cfd8ddc54bd6ab71fe0089d1bcea" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1ab26c14645918163bc48384cc67c" is_pre_election: true
I20260811 22:30:41.897060 22039 raft_consensus.cc:2454] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 0.
I20260811 22:30:41.897238 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.891136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:45697 (local address 127.18.211.129:44927)
0811 22:30:41.891519 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.891534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.891646 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.892148 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.892151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.892166 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:41.892306 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.892311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.893330 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.893333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.894556 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.894569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.896320 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.896323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.896396 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.896415 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.896981 (+ 566us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.897055 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":50,"thread_start_us":95,"threads_started":1}
I20260811 22:30:41.897473 21437 leader_election.cc:302] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3be78d4aef047bca9a2f721f2278021, bfb1ab26c14645918163bc48384cc67c; no voters:
I20260811 22:30:41.897789 22168 raft_consensus.cc:2781] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:41.897745 21796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e0cfd8ddc54bd6ab71fe0089d1bcea" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37260d27d1314235aeeefb62fb097f0e" is_pre_election: true
I20260811 22:30:41.897868 22168 raft_consensus.cc:482] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:41.897908 22168 raft_consensus.cc:3037] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:41.898008 21796 raft_consensus.cc:2454] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 0.
I20260811 22:30:41.899259 22168 raft_consensus.cc:504] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.899645 22168 leader_election.cc:288] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 election: Requested vote from peers 37260d27d1314235aeeefb62fb097f0e (127.18.211.130:45697), bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151)
I20260811 22:30:41.899999 21796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e0cfd8ddc54bd6ab71fe0089d1bcea" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37260d27d1314235aeeefb62fb097f0e"
I20260811 22:30:41.900125 21796 raft_consensus.cc:3037] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:41.900089 22039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e0cfd8ddc54bd6ab71fe0089d1bcea" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1ab26c14645918163bc48384cc67c"
I20260811 22:30:41.900218 22039 raft_consensus.cc:3037] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:41.901324 21796 raft_consensus.cc:2454] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 1.
I20260811 22:30:41.901317 22039 raft_consensus.cc:2454] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 1.
I20260811 22:30:41.901605 21438 leader_election.cc:302] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37260d27d1314235aeeefb62fb097f0e, a3be78d4aef047bca9a2f721f2278021; no voters:
I20260811 22:30:41.901813 22168 raft_consensus.cc:2781] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:41.902067 22168 raft_consensus.cc:686] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 1 LEADER]: Becoming Leader. State: Replica: a3be78d4aef047bca9a2f721f2278021, State: Running, Role: LEADER
I20260811 22:30:41.902346 22168 consensus_queue.cc:237] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.904906 21268 catalog_manager.cc:5495] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3be78d4aef047bca9a2f721f2278021 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:30:41 dist-test-slave-8hhm krb5kdc[21191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:41.956980 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.947118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.18.211.190:43827)
0811 22:30:41.947277 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:41.947282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:41.947572 (+ 290us) server_negotiation.cc:407] Connection header received
0811 22:30:41.947733 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:41.947735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:41.947776 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:41.947847 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:41.948776 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.949298 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.950413 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.950615 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.950705 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.953366 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:41.953369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:41.953371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:41.953391 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:41.955124 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.955595 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.955600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.955602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.955727 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:41.956056 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:41.956059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:41.956061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:41.956247 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:41.956358 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:41.956656 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:30:41.956829 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":66}
I20260811 22:30:41.956980 22220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:41.947031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43827 (local address 127.0.0.1:35880)
0811 22:30:41.947459 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:41.947476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:41.947626 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:41.947914 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:41.947917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:41.948493 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:41.948650 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.948659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.949410 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.949414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.950284 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:41.950296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:41.950745 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:41.950749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:41.950839 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:41.951456 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:41.951477 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:41.953153 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:41.955235 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.955247 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.955253 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.955489 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.955825 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:41.955828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:41.955830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:41.955966 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:41.956359 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:41.956366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:41.956485 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:30:41.956607 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 22:30:41.956770 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":153,"threads_started":1}
I20260811 22:30:41.961375 21268 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:35880:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:30:41.962172 21268 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:41.967478 21518 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1ba05ce6684d809e70291af8736692 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0a0d2708fe2435c9bf0b3c9498a5bc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:30:41.967541 21761 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1ba05ce6684d809e70291af8736692 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0a0d2708fe2435c9bf0b3c9498a5bc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:30:41.967917 21518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1ba05ce6684d809e70291af8736692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.967926 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1ba05ce6684d809e70291af8736692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.969650 22007 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1ba05ce6684d809e70291af8736692 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0a0d2708fe2435c9bf0b3c9498a5bc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:30:41.970070 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1ba05ce6684d809e70291af8736692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:41.970759 22159 tablet_bootstrap.cc:492] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: Bootstrap starting.
I20260811 22:30:41.971176 22160 tablet_bootstrap.cc:492] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: Bootstrap starting.
I20260811 22:30:41.971688 22159 tablet_bootstrap.cc:654] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.972293 22160 tablet_bootstrap.cc:654] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.973387 22159 tablet_bootstrap.cc:492] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: No bootstrap required, opened a new log
I20260811 22:30:41.973485 22159 ts_tablet_manager.cc:1397] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:30:41.973937 22159 raft_consensus.cc:348] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.974064 22159 raft_consensus.cc:374] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.974100 22159 raft_consensus.cc:729] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37260d27d1314235aeeefb62fb097f0e, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.974115 22160 tablet_bootstrap.cc:492] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: No bootstrap required, opened a new log
I20260811 22:30:41.974186 22160 ts_tablet_manager.cc:1397] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:30:41.974236 22159 consensus_queue.cc:260] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.974534 22159 ts_tablet_manager.cc:1428] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:41.974586 22160 raft_consensus.cc:348] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.974701 22160 raft_consensus.cc:374] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.974735 22160 raft_consensus.cc:729] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3be78d4aef047bca9a2f721f2278021, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.974781 22161 tablet_bootstrap.cc:492] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: Bootstrap starting.
I20260811 22:30:41.974849 22160 consensus_queue.cc:260] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.975081 22160 ts_tablet_manager.cc:1428] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:41.975339 22168 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:41.975464 22168 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:41.975811 22161 tablet_bootstrap.cc:654] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:41.976630 22167 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:41.976769 22167 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:41.977811 22161 tablet_bootstrap.cc:492] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: No bootstrap required, opened a new log
I20260811 22:30:41.977908 22161 ts_tablet_manager.cc:1397] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:30:41.978432 22161 raft_consensus.cc:348] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.978555 22161 raft_consensus.cc:374] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:41.978601 22161 raft_consensus.cc:729] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb1ab26c14645918163bc48384cc67c, State: Initialized, Role: FOLLOWER
I20260811 22:30:41.978729 22161 consensus_queue.cc:260] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:41.979010 22161 ts_tablet_manager.cc:1428] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:41.978969 22169 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:41.979166 22169 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:42.117980 22168 raft_consensus.cc:482] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:42.118126 22168 raft_consensus.cc:504] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:42.118624 22168 leader_election.cc:288] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37260d27d1314235aeeefb62fb097f0e (127.18.211.130:45697), bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151)
I20260811 22:30:42.119118 22039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1ba05ce6684d809e70291af8736692" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1ab26c14645918163bc48384cc67c" is_pre_election: true
I20260811 22:30:42.119104 21796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1ba05ce6684d809e70291af8736692" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37260d27d1314235aeeefb62fb097f0e" is_pre_election: true
I20260811 22:30:42.119293 21796 raft_consensus.cc:2454] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 0.
I20260811 22:30:42.119302 22039 raft_consensus.cc:2454] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 0.
I20260811 22:30:42.119660 21438 leader_election.cc:302] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37260d27d1314235aeeefb62fb097f0e, a3be78d4aef047bca9a2f721f2278021; no voters:
I20260811 22:30:42.119851 22168 raft_consensus.cc:2781] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:42.119915 22168 raft_consensus.cc:482] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:42.119972 22168 raft_consensus.cc:3037] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:42.120978 22168 raft_consensus.cc:504] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:42.121449 22168 leader_election.cc:288] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 election: Requested vote from peers 37260d27d1314235aeeefb62fb097f0e (127.18.211.130:45697), bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151)
I20260811 22:30:42.121800 21796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1ba05ce6684d809e70291af8736692" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37260d27d1314235aeeefb62fb097f0e"
I20260811 22:30:42.121807 22039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1ba05ce6684d809e70291af8736692" candidate_uuid: "a3be78d4aef047bca9a2f721f2278021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1ab26c14645918163bc48384cc67c"
I20260811 22:30:42.121929 21796 raft_consensus.cc:3037] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:42.121953 22039 raft_consensus.cc:3037] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:42.122874 22039 raft_consensus.cc:2454] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 1.
I20260811 22:30:42.123116 21796 raft_consensus.cc:2454] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3be78d4aef047bca9a2f721f2278021 in term 1.
I20260811 22:30:42.123176 21437 leader_election.cc:302] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3be78d4aef047bca9a2f721f2278021, bfb1ab26c14645918163bc48384cc67c; no voters:
I20260811 22:30:42.123354 22168 raft_consensus.cc:2781] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:42.123441 22168 raft_consensus.cc:686] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [term 1 LEADER]: Becoming Leader. State: Replica: a3be78d4aef047bca9a2f721f2278021, State: Running, Role: LEADER
I20260811 22:30:42.123589 22168 consensus_queue.cc:237] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } }
I20260811 22:30:42.124357 22209 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: TxnStatusTablet state changed. Reason: New leader a3be78d4aef047bca9a2f721f2278021. Latest consensus state: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:42.124480 22209 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:30:42.124854 22228 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:30:42.125023 22228 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:30:42.125567 21267 catalog_manager.cc:5495] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3be78d4aef047bca9a2f721f2278021 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } health_report { overall_health: UNKNOWN } } }
I20260811 22:30:42.174460 22220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:42.170261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41669 (local address 127.0.0.1:38332)
0811 22:30:42.170476 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:42.170493 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:42.170643 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:42.171003 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:42.171006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:42.171014 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:30:42.171125 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:42.171132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.171946 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.171949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:42.172741 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:42.172752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.173284 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.173287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:42.173377 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:42.173435 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:30:42.174162 (+ 727us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:30:42.174180 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:30:42.174252 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:30:42.174346 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:30:42.174460 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:42.170405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.18.211.129:41669)
0811 22:30:42.170560 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:42.170564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:42.170580 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:30:42.170813 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:42.170816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:42.170880 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:42.170991 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:42.171250 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.171824 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.172869 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.173107 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.173301 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:42.173594 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:42.174049 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:42.174191 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:42.174261 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:30:42.174313 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:30:42.177951 22039 raft_consensus.cc:1261] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Refusing update from remote peer a3be78d4aef047bca9a2f721f2278021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:42.178143 21796 raft_consensus.cc:1261] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Refusing update from remote peer a3be78d4aef047bca9a2f721f2278021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:42.178504 22209 consensus_queue.cc:1035] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:42.178928 22209 consensus_queue.cc:1035] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:42.180481 22169 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: TxnStatusTablet state changed. Reason: New leader a3be78d4aef047bca9a2f721f2278021. Latest consensus state: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:42.180595 22169 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:42.180603 22167 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: TxnStatusTablet state changed. Reason: New leader a3be78d4aef047bca9a2f721f2278021. Latest consensus state: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:42.180720 22167 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:42.182631 22209 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:42.182742 22209 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:30:42.183080 22177 mvcc.cc:205] Tried to move back new op lower bound from 7317452563155406848 to 7317452562939809792. Current Snapshot: MvccSnapshot[applied={T|T < 7317452563155406848}]
I20260811 22:30:42.183267 22167 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:42.183387 22167 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P 37260d27d1314235aeeefb62fb097f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:42.183816 22209 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:42.183926 22209 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P a3be78d4aef047bca9a2f721f2278021: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:30:42.185475 22169 tablet_replica.cc:439] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3be78d4aef047bca9a2f721f2278021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3be78d4aef047bca9a2f721f2278021" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41669 } } peers { permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 } } peers { permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 } } }
I20260811 22:30:42.185652 22169 tablet_replica.cc:442] T 1c1ba05ce6684d809e70291af8736692 P bfb1ab26c14645918163bc48384cc67c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:42.194336 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:42.190781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.18.211.129:41669)
0811 22:30:42.190914 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:42.190918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:42.190996 (+ 78us) server_negotiation.cc:407] Connection header received
0811 22:30:42.191133 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:42.191136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:42.191175 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:42.191235 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:42.191528 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.192043 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.193205 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.193521 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.193613 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:42.193905 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:42.193970 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:42.194021 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:42.194141 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 22:30:42.194185 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:30:42.205545 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:42.201908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:55837 (local address 127.18.211.129:41669)
0811 22:30:42.202038 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:42.202043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:42.202244 (+ 201us) server_negotiation.cc:407] Connection header received
0811 22:30:42.202349 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:42.202351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:42.202392 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:42.202455 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:42.202752 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.203255 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.204489 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.204685 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.204761 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:42.205105 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:42.205173 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:42.205265 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:42.205358 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 22:30:42.205405 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:30:42.205608 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:42.201838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41669 (local address 127.18.211.129:55837)
0811 22:30:42.202148 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:42.202163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:42.202272 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:42.202491 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:42.202494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:42.202501 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:30:42.202639 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:42.202645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.203366 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.203369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:42.204370 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:42.204381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.204809 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.204813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:42.204907 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:42.204967 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:30:42.205272 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:30:42.205279 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:30:42.205349 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:30:42.205428 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":115,"threads_started":1}
I20260811 22:30:42.213095 22039 raft_consensus.cc:1261] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Refusing update from remote peer a3be78d4aef047bca9a2f721f2278021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:42.213142 21796 raft_consensus.cc:1261] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Refusing update from remote peer a3be78d4aef047bca9a2f721f2278021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:42.213529 22241 consensus_queue.cc:1035] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1ab26c14645918163bc48384cc67c" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:42.213726 22209 consensus_queue.cc:1035] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37260d27d1314235aeeefb62fb097f0e" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:42.217320 22179 mvcc.cc:205] Tried to move back new op lower bound from 7317452563301158912 to 7317452562034753536. Current Snapshot: MvccSnapshot[applied={T|T < 7317452563301158912}]
I20260811 22:30:42.255837 21272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:42.256038 21272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:42.258438 21272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 321ca9c11eaf44fe8989909ff7ecc7aa: Sending DeleteTablet for 3 replicas of tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea
I20260811 22:30:42.259070 21518 tablet_service.cc:1514] Processing DeleteTablet for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:42 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:38316
I20260811 22:30:42.259298 22192 tablet_replica.cc:330] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: stopping tablet replica
I20260811 22:30:42.259439 22192 raft_consensus.cc:2229] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:30:42.259799 22192 raft_consensus.cc:2258] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:42.260257 21761 tablet_service.cc:1514] Processing DeleteTablet for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:42 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:58600
I20260811 22:30:42.260490 22195 tablet_replica.cc:330] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: stopping tablet replica
I20260811 22:30:42.260628 22195 raft_consensus.cc:2229] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:42.260754 22195 raft_consensus.cc:2258] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:42.260978 22192 ts_tablet_manager.cc:1905] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:42.264263 22192 ts_tablet_manager.cc:1918] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:42.264362 22192 log.cc:1196] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/d8e0cfd8ddc54bd6ab71fe0089d1bcea
I20260811 22:30:42.264659 22192 ts_tablet_manager.cc:1939] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P a3be78d4aef047bca9a2f721f2278021: Deleting consensus metadata
I20260811 22:30:42.265506 21244 catalog_manager.cc:4841] TS a3be78d4aef047bca9a2f721f2278021 (127.18.211.129:41669): tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea (table test-table [id=3343329fe87e48a59641ea58d44b1540]) successfully deleted
I20260811 22:30:42.265985 22007 tablet_service.cc:1514] Processing DeleteTablet for tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:42 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56858
I20260811 22:30:42.266234 22191 tablet_replica.cc:330] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: stopping tablet replica
I20260811 22:30:42.266307 22195 ts_tablet_manager.cc:1905] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:42.266377 22191 raft_consensus.cc:2229] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:42.266501 22191 raft_consensus.cc:2258] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:42.267355 22191 ts_tablet_manager.cc:1905] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:42.269405 22195 ts_tablet_manager.cc:1918] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:42.269500 22195 log.cc:1196] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/d8e0cfd8ddc54bd6ab71fe0089d1bcea
I20260811 22:30:42.269809 22195 ts_tablet_manager.cc:1939] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P 37260d27d1314235aeeefb62fb097f0e: Deleting consensus metadata
I20260811 22:30:42.270093 22191 ts_tablet_manager.cc:1918] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:42.270190 22191 log.cc:1196] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/d8e0cfd8ddc54bd6ab71fe0089d1bcea
I20260811 22:30:42.270462 22191 ts_tablet_manager.cc:1939] T d8e0cfd8ddc54bd6ab71fe0089d1bcea P bfb1ab26c14645918163bc48384cc67c: Deleting consensus metadata
I20260811 22:30:42.270561 21244 catalog_manager.cc:4841] TS 37260d27d1314235aeeefb62fb097f0e (127.18.211.130:45697): tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea (table test-table [id=3343329fe87e48a59641ea58d44b1540]) successfully deleted
I20260811 22:30:42.271248 21243 catalog_manager.cc:4841] TS bfb1ab26c14645918163bc48384cc67c (127.18.211.131:44151): tablet d8e0cfd8ddc54bd6ab71fe0089d1bcea (table test-table [id=3343329fe87e48a59641ea58d44b1540]) successfully deleted
Aug 11 22:30:42 dist-test-slave-8hhm krb5kdc[21191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487441, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:30:42.280889 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:42.267886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.18.211.129:41669)
0811 22:30:42.269380 (+ 1494us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:42.269385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:42.269930 (+ 545us) server_negotiation.cc:407] Connection header received
0811 22:30:42.270181 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:42.270185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:42.270244 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:42.270307 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:42.271563 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.272130 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.273103 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.273415 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.273524 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:42.276734 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:42.276744 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:42.276761 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:42.276789 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:42.278921 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:42.279361 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:42.279368 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:42.279370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:42.279490 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:42.279866 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:42.279870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:42.279872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:42.280185 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:42.280416 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:42.280587 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:30:42.280726 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1405}
W20260811 22:30:42.281397 21632 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38350
I20260811 22:30:42.291610 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:42.283298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.18.211.190:43827)
0811 22:30:42.283425 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:42.283429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:42.283494 (+ 65us) server_negotiation.cc:407] Connection header received
0811 22:30:42.283606 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:42.283609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:42.283656 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:42.283746 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:42.284472 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.285331 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.286103 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:42.286471 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:42.286583 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:42.287879 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:42.287883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:42.287886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:42.287916 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:42.289619 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:42.290070 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:42.290074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:42.290075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:42.290172 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:42.290603 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:42.290608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:42.290610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:42.290831 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:42.291051 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:42.291284 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 22:30:42.291419 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:30:42.292107 21272 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884
I20260811 22:30:42.293730 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21404
I20260811 22:30:42.309840 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21656
I20260811 22:30:42.334044 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21892
I20260811 22:30:42.352035 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21216
2026-08-11T22:30:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3179 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:42 dist-test-slave-8hhm krb5kdc[22259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:42 dist-test-slave-8hhm krb5kdc[22259](info): set up 2 sockets
Aug 11 22:30:42 dist-test-slave-8hhm krb5kdc[22259](info): commencing operation
krb5kdc: starting...
W20260811 22:30:43.738482 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:43 dist-test-slave-8hhm krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:43.880448 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39385
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:42099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:39385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:44.003415 22284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:44.003759 22284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:44.003844 22284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:44.008081 22284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:44.008183 22284 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:30:44.008214 22284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:44.008258 22284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:44.008293 22284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:44.008347 22284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:44.013998 22284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:42099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:39385
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:44.015166 22284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:44.016239 22284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:30:44.025417 22284 server_base.cc:1034] running on GCE node
W20260811 22:30:44.025357 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:44.025352 22290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:44.025358 22294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:44.026281 22284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:44.027463 22284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:44.028668 22284 hybrid_clock.cc:648] HybridClock initialized: now 1786487444028646 us; error 32 us; skew 500 ppm
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:44.037710 22284 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:44.038863 22284 webserver.cc:460] Webserver started at http://127.18.211.190:43549/ using document root <none> and password file <none>
I20260811 22:30:44.039446 22284 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:44.039510 22284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:44.039719 22284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:44.042032 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "99097b5118ad4c3e9f2fd6b4f74e0416"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "868cfd525067263d2485e3a120d6ff96"
server_key_iv: "3fe6b5c599be805bd81f8b88ae12f8cf"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.042615 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "99097b5118ad4c3e9f2fd6b4f74e0416"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "868cfd525067263d2485e3a120d6ff96"
server_key_iv: "3fe6b5c599be805bd81f8b88ae12f8cf"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.046466 22284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:30:44.048950 22309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:44.050359 22284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:30:44.050503 22284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "99097b5118ad4c3e9f2fd6b4f74e0416"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "868cfd525067263d2485e3a120d6ff96"
server_key_iv: "3fe6b5c599be805bd81f8b88ae12f8cf"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.050621 22284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:44.062727 22284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:44.066200 22284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:44.066454 22284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:44.075498 22284 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:39385
I20260811 22:30:44.075505 22411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:39385 every 8 connection(s)
I20260811 22:30:44.076627 22284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:44.079099 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22284
I20260811 22:30:44.079249 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:44.079573 19278 external_mini_cluster.cc:1428] Setting key aca6d7787a4d0c170eafc98b0afcd5bc
I20260811 22:30:44.079811 22412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:44.087282 22412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Bootstrap starting.
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:44.090270 22412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:44.091017 22412 log.cc:824] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:44.093005 22412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: No bootstrap required, opened a new log
I20260811 22:30:44.096094 22412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39385 } }
I20260811 22:30:44.096369 22412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:44.096454 22412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99097b5118ad4c3e9f2fd6b4f74e0416, State: Initialized, Role: FOLLOWER
I20260811 22:30:44.096503 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.081116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.18.211.190:39385)
0811 22:30:44.081669 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.081688 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.081728 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:30:44.082791 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.082813 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.083174 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.083482 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.084516 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.085627 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.086815 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.087375 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.087595 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.090596 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.090601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.090616 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.090647 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.093368 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.093871 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.093878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.093884 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.094020 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.094341 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.094346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.094348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.094772 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.095001 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.095307 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.095626 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":164,"threads_started":1}
I20260811 22:30:44.096988 22412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39385 } }
I20260811 22:30:44.097155 22412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:44.097267 22412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:44.097383 22412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:44.098307 22412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39385 } }
I20260811 22:30:44.098682 22412 leader_election.cc:302] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99097b5118ad4c3e9f2fd6b4f74e0416; no voters:
I20260811 22:30:44.099066 22412 leader_election.cc:288] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:44.099150 22421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:44.099403 22421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [term 1 LEADER]: Becoming Leader. State: Replica: 99097b5118ad4c3e9f2fd6b4f74e0416, State: Running, Role: LEADER
I20260811 22:30:44.099718 22421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39385 } }
I20260811 22:30:44.100311 22412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:44.101490 22423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99097b5118ad4c3e9f2fd6b4f74e0416. Latest consensus state: current_term: 1 leader_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39385 } } }
I20260811 22:30:44.101634 22423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:44.101912 22422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99097b5118ad4c3e9f2fd6b4f74e0416" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39385 } } }
I20260811 22:30:44.102006 22422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:44.102134 22443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:44.105823 22443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:44.112295 22443 catalog_manager.cc:1347] Generated new cluster ID: ffb2486015b04425b0436c2dd52f4980
I20260811 22:30:44.112378 22443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:44.132814 22443 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:44.134243 22443 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:44.148088 22443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Generated new TSK 0
I20260811 22:30:44.148988 22443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:30:44.184568 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39385
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:42099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:44.306056 22457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:44.306381 22457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:44.306432 22457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:44.310457 22457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:44.310523 22457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:44.310563 22457 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:30:44.310578 22457 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:30:44.310611 22457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:30:44.316009 22457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:42099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:39385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:44.317126 22457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:44.318523 22457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:44.329525 22469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:44.329548 22466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:44.329535 22467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:44.329583 22457 server_base.cc:1034] running on GCE node
I20260811 22:30:44.330237 22457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:44.331032 22457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:44.332212 22457 hybrid_clock.cc:648] HybridClock initialized: now 1786487444332171 us; error 54 us; skew 500 ppm
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:44.335446 22457 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:30:44.336426 22457 webserver.cc:460] Webserver started at http://127.18.211.129:36209/ using document root <none> and password file <none>
I20260811 22:30:44.337004 22457 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:44.337066 22457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:44.337353 22457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:44.339572 22457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "3d646afff3dd40a894fbdddf4da617a0"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "7dcb1d4543a65c78a3701943fd28c0e5"
server_key_iv: "e5c5aab18abe57bb028607e9a1d6688e"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.340138 22457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "3d646afff3dd40a894fbdddf4da617a0"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "7dcb1d4543a65c78a3701943fd28c0e5"
server_key_iv: "e5c5aab18abe57bb028607e9a1d6688e"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.344246 22457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:30:44.346916 22482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:44.348312 22457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 22:30:44.348488 22457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "3d646afff3dd40a894fbdddf4da617a0"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "7dcb1d4543a65c78a3701943fd28c0e5"
server_key_iv: "e5c5aab18abe57bb028607e9a1d6688e"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.348639 22457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:44.358827 22457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:44.362547 22457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:44.362785 22457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:44.364450 22457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:44.364545 22457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:44.364617 22457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:44.364668 22457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:44.376427 22457 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:34929
I20260811 22:30:44.376746 22683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:34929 every 8 connection(s)
I20260811 22:30:44.377615 22457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:30:44.379631 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.365505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:33877 (local address 127.18.211.190:39385)
0811 22:30:44.365647 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.365651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.366892 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 22:30:44.368244 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.368248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.368318 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.368414 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.370335 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.370983 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.372315 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.372557 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.372694 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.375505 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.375509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.375512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.375545 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.377264 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.377828 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.377832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.377834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.377944 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.378476 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.378481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.378484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.378787 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.378902 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.379382 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.379505 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:30:44.380517 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.366161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39385 (local address 127.18.211.129:33877)
0811 22:30:44.366718 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.366765 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.367924 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.368672 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.368683 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.369298 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:44.370129 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.370156 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.371156 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.371160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.372161 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.372181 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.372746 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.372751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.372940 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.373650 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:44.373667 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:44.375318 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:44.377393 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.377400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.377416 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.377690 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.378154 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.378157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.378159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.378357 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.378902 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:44.378908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:44.379143 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.379322 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.379589 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":159,"threads_started":1}
I20260811 22:30:44.381985 22684 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39385
I20260811 22:30:44.382143 22684 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:44.382793 22684 heartbeater.cc:511] Master 127.18.211.190:39385 requested a full tablet report, sending...
I20260811 22:30:44.383757 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22457
I20260811 22:30:44.383903 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:30:44.384229 19278 external_mini_cluster.cc:1428] Setting key 57e1376f698c7652895a3369d702eacf
I20260811 22:30:44.384675 22333 ts_manager.cc:194] Registered new tserver with Master: 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929)
I20260811 22:30:44.386174 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:33877
I20260811 22:30:44.393230 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.384957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:49623 (local address 127.18.211.190:39385)
0811 22:30:44.385066 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.385070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.385281 (+ 211us) server_negotiation.cc:407] Connection header received
0811 22:30:44.385455 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.385458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.385512 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.385602 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.386167 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.386856 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.388071 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.388393 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.388470 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.390025 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.390028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.390031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.390062 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.391489 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.391960 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.391964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.391966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.392051 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.392353 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.392358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.392360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.392541 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.392655 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.392924 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.393041 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:30:44.393275 22695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.384859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39385 (local address 127.18.211.129:49623)
0811 22:30:44.385166 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.385179 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.385351 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.385620 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.385623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.385889 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:44.386027 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.386036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.387043 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.387049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.387925 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.387937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.388629 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.388633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.388717 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.389406 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:44.389421 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:44.389884 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:44.391612 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.391617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.391620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.391852 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.392151 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.392156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.392158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.392259 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.392645 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:44.392648 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:44.392754 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.392866 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.393023 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":3,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:30:44.416985 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39385
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:42099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:44.539189 22698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:44.539505 22698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:44.539554 22698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:44.543763 22698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:44.543851 22698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:44.543941 22698 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:30:44.543977 22698 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:30:44.544044 22698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:30:44.550179 22698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:42099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:39385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:44.551363 22698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:44.552572 22698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:44.563983 22712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:44.564075 22698 server_base.cc:1034] running on GCE node
W20260811 22:30:44.564052 22718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:44.563983 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:44.564872 22698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:44.565718 22698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:44.566924 22698 hybrid_clock.cc:648] HybridClock initialized: now 1786487444566908 us; error 32 us; skew 500 ppm
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:44.570282 22698 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:30:44.571286 22698 webserver.cc:460] Webserver started at http://127.18.211.130:41371/ using document root <none> and password file <none>
I20260811 22:30:44.572014 22698 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:44.572088 22698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:44.572301 22698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:44.574662 22698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "f480fbc8cc734489b4790d07d613435c"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "84ee1e05c6a6f1e50ed29a36c08a0004"
server_key_iv: "919922fb8028fefd1a2ae975afa43597"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.575246 22698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "f480fbc8cc734489b4790d07d613435c"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "84ee1e05c6a6f1e50ed29a36c08a0004"
server_key_iv: "919922fb8028fefd1a2ae975afa43597"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.579382 22698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:44.582188 22733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:44.583660 22698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:30:44.583772 22698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "f480fbc8cc734489b4790d07d613435c"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "84ee1e05c6a6f1e50ed29a36c08a0004"
server_key_iv: "919922fb8028fefd1a2ae975afa43597"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.583875 22698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:44.602816 22698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:44.606156 22698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:44.606396 22698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:44.607946 22698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:44.607998 22698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:44.608042 22698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:44.608103 22698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:44.619745 22698 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:43791
I20260811 22:30:44.619819 22928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:43791 every 8 connection(s)
I20260811 22:30:44.620800 22698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:30:44.623035 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.608850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:57915 (local address 127.18.211.190:39385)
0811 22:30:44.608994 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.608998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.610454 (+ 1456us) server_negotiation.cc:407] Connection header received
0811 22:30:44.611755 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.611762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.611839 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.611950 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.613604 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.614141 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.615474 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.615869 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.615939 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.618905 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.618909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.618912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.618949 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.620612 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.621306 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.621312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.621317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.621421 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.621875 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.621879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.621880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.622159 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.622271 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.622802 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.622913 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:30:44.623862 22744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.609710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39385 (local address 127.18.211.130:57915)
0811 22:30:44.610274 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.610319 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.611417 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.612194 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.612207 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.612686 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:44.613439 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.613457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.614281 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.614285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.615296 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.615311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.616138 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.616140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.616284 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.616966 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:44.616983 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:44.618713 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:44.620746 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.620752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.620767 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.621132 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.621558 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.621561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.621563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.621745 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.622282 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:44.622288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:44.622555 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.622710 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.622971 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":150,"threads_started":1}
I20260811 22:30:44.625128 22931 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39385
I20260811 22:30:44.625349 22931 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:44.625573 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22698
I20260811 22:30:44.625697 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:30:44.625972 22931 heartbeater.cc:511] Master 127.18.211.190:39385 requested a full tablet report, sending...
I20260811 22:30:44.626022 19278 external_mini_cluster.cc:1428] Setting key aec4342fec8cdbcf24f8b01ceaa02a2e
I20260811 22:30:44.626984 22333 ts_manager.cc:194] Registered new tserver with Master: f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791)
I20260811 22:30:44.627549 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:57915
I20260811 22:30:44.635386 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.627684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:40945 (local address 127.18.211.190:39385)
0811 22:30:44.627799 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.627805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.627995 (+ 190us) server_negotiation.cc:407] Connection header received
0811 22:30:44.628119 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.628124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.628177 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.628280 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.628743 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.629408 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.630230 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.630452 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.630574 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.631757 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.631765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.631768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.631791 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.633434 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.633971 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.633975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.633977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.634072 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.634382 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.634387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.634389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.634646 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.634774 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.635059 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.635183 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 22:30:44.635401 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.627611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39385 (local address 127.18.211.130:40945)
0811 22:30:44.627902 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.627918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.628018 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.628297 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.628300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.628529 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:44.628624 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.628629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.629542 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.629545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.630104 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.630113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.630579 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.630583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.630677 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.631240 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:44.631251 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:44.631596 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:44.633617 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.633621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.633623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.633854 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.634180 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.634185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.634187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.634279 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.634761 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:44.634764 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:44.634878 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.634998 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.635183 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:30:44.660095 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39385
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:42099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:44.780189 22945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:44.780488 22945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:44.780539 22945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:44.784821 22945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:44.784899 22945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:44.784941 22945 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:30:44.785000 22945 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:30:44.785061 22945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:30:44.790928 22945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:42099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:39385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:44.792234 22945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:44.793416 22945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:44.804286 22962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:44.804476 22945 server_base.cc:1034] running on GCE node
W20260811 22:30:44.804272 22957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:44.804267 22960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:44.805148 22945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:44.805902 22945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:44.807088 22945 hybrid_clock.cc:648] HybridClock initialized: now 1786487444807073 us; error 34 us; skew 500 ppm
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:44.810312 22945 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:30:44.811288 22945 webserver.cc:460] Webserver started at http://127.18.211.131:33749/ using document root <none> and password file <none>
I20260811 22:30:44.811949 22945 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:44.812059 22945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:44.812317 22945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:44.814601 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "f44cd316376c45ceb802c8828663b674"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "c758a850f15ece072e1793451f615389"
server_key_iv: "bcb30f5101f614f3c2a6e9f5f81ca06e"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.815258 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "f44cd316376c45ceb802c8828663b674"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "c758a850f15ece072e1793451f615389"
server_key_iv: "bcb30f5101f614f3c2a6e9f5f81ca06e"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.819535 22945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:44.822324 22981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:44.823715 22945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:44.823873 22945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "f44cd316376c45ceb802c8828663b674"
format_stamp: "Formatted at 2026-08-11 22:30:44 on dist-test-slave-8hhm"
server_key: "c758a850f15ece072e1793451f615389"
server_key_iv: "bcb30f5101f614f3c2a6e9f5f81ca06e"
server_key_version: "encryptionkey@0"
I20260811 22:30:44.824012 22945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:44.844905 22945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:44.848201 22945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:44.848448 22945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:44.850320 22945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:44.850392 22945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:44.850447 22945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:44.850469 22945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:44.863176 22945 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:33391
I20260811 22:30:44.863219 23194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:33391 every 8 connection(s)
I20260811 22:30:44.864243 22945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:30:44.866475 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.851167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:47527 (local address 127.18.211.190:39385)
0811 22:30:44.851343 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.851349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.852516 (+ 1167us) server_negotiation.cc:407] Connection header received
0811 22:30:44.853912 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.853916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.853989 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.854105 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.855649 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.856275 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.858017 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.858429 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.858575 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.862247 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.862250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.862252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.862281 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.864120 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.864806 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.864810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.864812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.864913 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.865375 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.865379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.865380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.865620 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.865715 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.866240 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.866351 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:30:44.867370 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.851679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39385 (local address 127.18.211.131:47527)
0811 22:30:44.852312 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.852372 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.853623 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.854339 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.854352 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.854857 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:44.855480 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.855508 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.856433 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.856441 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.857802 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.857813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.858637 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.858642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.858897 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.859705 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:44.859725 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:44.862004 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:44.864263 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.864270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.864286 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.864656 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.865035 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.865039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.865041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.865254 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.865747 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:44.865752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:44.865996 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.866158 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.866431 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":238,"threads_started":1}
I20260811 22:30:44.868394 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22945
I20260811 22:30:44.868532 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:30:44.868877 19278 external_mini_cluster.cc:1428] Setting key ed72827adb74e42d043db96f354b79a3
I20260811 22:30:44.868978 23196 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39385
I20260811 22:30:44.869155 23196 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:44.869830 23196 heartbeater.cc:511] Master 127.18.211.190:39385 requested a full tablet report, sending...
I20260811 22:30:44.870873 22333 ts_manager.cc:194] Registered new tserver with Master: f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:44.871609 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:47527
I20260811 22:30:44.878353 23205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.871288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39385 (local address 127.18.211.131:40837)
0811 22:30:44.871562 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.871581 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.871668 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.872131 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.872134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.872365 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:44.872512 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.872519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.873319 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.873323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.874124 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.874133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.874538 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.874541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.874633 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.875106 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:44.875117 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:44.875439 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:44.876974 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.876976 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.876978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.877179 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.877512 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:44.877514 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:44.877515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:44.877606 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:44.877949 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:44.877950 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:44.878027 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.878109 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.878217 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":110,"threads_started":1}
I20260811 22:30:44.878391 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.871354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:40837 (local address 127.18.211.190:39385)
0811 22:30:44.871468 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.871472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.871658 (+ 186us) server_negotiation.cc:407] Connection header received
0811 22:30:44.871957 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.871961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.872011 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.872114 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.872645 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.873141 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.874233 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.874439 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.874517 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.875566 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.875569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.875571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.875594 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.876874 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.877344 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.877347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.877348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.877427 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.877699 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.877701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.877703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.877865 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.877941 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.878184 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.878282 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:30:44.884500 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:30:44 dist-test-slave-8hhm krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:44.915243 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.906975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38158 (local address 127.18.211.190:39385)
0811 22:30:44.907127 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.907131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.907144 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:30:44.907188 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.907190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.907231 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.907299 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:44.908033 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.908576 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.909506 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.909731 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.909813 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.912354 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:44.912357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:44.912359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:44.912387 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:44.913759 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.914170 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.914173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.914175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.914253 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:44.914495 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:44.914497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:44.914499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:44.914692 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:44.914794 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.915008 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.915111 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:30:44.918324 22333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:30:44.921522 22333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:44.937109 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.930779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:43791 (local address 127.0.0.1:55480)
0811 22:30:44.931151 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.931211 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.931355 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.933099 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.933102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.933119 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:44.933282 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.933289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.934327 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.934332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.935651 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.935662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.936519 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.936523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.936644 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.936696 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.936813 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.936945 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260811 22:30:44.937162 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.930992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.18.211.130:43791)
0811 22:30:44.932615 (+ 1623us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.932623 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.932643 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:30:44.932709 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.932712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.932911 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.933116 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:44.933413 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.934200 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.935790 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.936383 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.936554 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.936718 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.936853 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.936953 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1532,"thread_start_us":144,"threads_started":1}
I20260811 22:30:44.938056 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.931156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:33391 (local address 127.0.0.1:56186)
0811 22:30:44.931573 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.931588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.931676 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.932900 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.932906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.932931 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:44.933119 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.933129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.934684 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.934690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.936442 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.936448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.937607 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.937611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.937734 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.937765 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.937848 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.937921 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":982,"thread_start_us":126,"threads_started":1}
I20260811 22:30:44.938444 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.932152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56186 (local address 127.18.211.131:33391)
0811 22:30:44.932488 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.932499 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.932517 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:30:44.932589 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.932593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.932735 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.932889 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:44.933295 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.934537 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.936570 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.937438 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.937627 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.937751 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.938056 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.938295 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":128,"threads_started":1}
I20260811 22:30:44.938771 23234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.931567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34929 (local address 127.0.0.1:56528)
0811 22:30:44.931864 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:44.931875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:44.931972 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:44.933273 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:44.933278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:44.933300 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:44.933416 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.933423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.934757 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.934760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.935751 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:44.935762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.938302 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.938306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:44.938390 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.938420 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:30:44.938495 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:30:44.938646 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":2,"thread_start_us":137,"threads_started":1}
I20260811 22:30:44.939188 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:44.931797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.18.211.129:34929)
0811 22:30:44.932367 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:44.932375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:44.932403 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:30:44.932488 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:44.932496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:44.932723 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:44.932889 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:44.933554 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.934639 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.936648 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:44.937438 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:44.937621 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:44.937752 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:44.938948 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0811 22:30:44.939054 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":101,"threads_started":1}
I20260811 22:30:44.940667 22816 tablet_service.cc:1467] Processing CreateTablet for tablet 49f6838fdac24c93b0a7fb92545deb66 (DEFAULT_TABLE table=test-table [id=731e334414694a09b4455093a89333df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:44.940861 23068 tablet_service.cc:1467] Processing CreateTablet for tablet 49f6838fdac24c93b0a7fb92545deb66 (DEFAULT_TABLE table=test-table [id=731e334414694a09b4455093a89333df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:44.941864 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f6838fdac24c93b0a7fb92545deb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:44.941907 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f6838fdac24c93b0a7fb92545deb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:44.944125 22569 tablet_service.cc:1467] Processing CreateTablet for tablet 49f6838fdac24c93b0a7fb92545deb66 (DEFAULT_TABLE table=test-table [id=731e334414694a09b4455093a89333df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:44.945194 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f6838fdac24c93b0a7fb92545deb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:44.949378 23238 tablet_bootstrap.cc:492] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Bootstrap starting.
I20260811 22:30:44.951630 23238 tablet_bootstrap.cc:654] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:44.952744 23239 tablet_bootstrap.cc:492] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Bootstrap starting.
I20260811 22:30:44.952638 23238 log.cc:824] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:44.953769 23245 tablet_bootstrap.cc:492] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Bootstrap starting.
I20260811 22:30:44.955492 23239 tablet_bootstrap.cc:654] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:44.955781 23238 tablet_bootstrap.cc:492] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: No bootstrap required, opened a new log
I20260811 22:30:44.955951 23245 tablet_bootstrap.cc:654] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:44.956053 23238 ts_tablet_manager.cc:1397] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:30:44.956491 23239 log.cc:824] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:44.956795 23245 log.cc:824] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:44.958792 23245 tablet_bootstrap.cc:492] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: No bootstrap required, opened a new log
I20260811 22:30:44.958792 23239 tablet_bootstrap.cc:492] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: No bootstrap required, opened a new log
I20260811 22:30:44.959009 23245 ts_tablet_manager.cc:1397] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:30:44.959009 23239 ts_tablet_manager.cc:1397] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:30:44.959638 23238 raft_consensus.cc:348] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:44.959980 23238 raft_consensus.cc:374] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:44.960031 23238 raft_consensus.cc:729] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44cd316376c45ceb802c8828663b674, State: Initialized, Role: FOLLOWER
I20260811 22:30:44.960534 23238 consensus_queue.cc:260] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:44.961668 23238 ts_tablet_manager.cc:1428] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:30:44.962106 23196 heartbeater.cc:503] Master 127.18.211.190:39385 was elected leader, sending a full tablet report...
I20260811 22:30:44.963548 23245 raft_consensus.cc:348] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:44.963873 23245 raft_consensus.cc:374] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:44.963721 23239 raft_consensus.cc:348] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:44.963933 23245 raft_consensus.cc:729] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f480fbc8cc734489b4790d07d613435c, State: Initialized, Role: FOLLOWER
I20260811 22:30:44.963943 23239 raft_consensus.cc:374] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:44.964003 23239 raft_consensus.cc:729] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Initialized, Role: FOLLOWER
I20260811 22:30:44.964439 23245 consensus_queue.cc:260] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:44.964439 23239 consensus_queue.cc:260] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:44.965181 22684 heartbeater.cc:503] Master 127.18.211.190:39385 was elected leader, sending a full tablet report...
I20260811 22:30:44.965550 23239 ts_tablet_manager.cc:1428] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 22:30:44.965550 23245 ts_tablet_manager.cc:1428] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:30:44.965907 22931 heartbeater.cc:503] Master 127.18.211.190:39385 was elected leader, sending a full tablet report...
I20260811 22:30:45.058846 23254 raft_consensus.cc:482] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:45.059126 23254 raft_consensus.cc:504] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.060433 23254 leader_election.cc:288] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:45.064467 22744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.060737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:33391 (local address 127.18.211.130:54297)
0811 22:30:45.060867 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.060884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.060986 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.061308 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.061311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.061337 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:45.061465 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.061475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.062392 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.062397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.063391 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.063400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.064050 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.064054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.064186 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.064222 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.064285 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.064351 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 22:30:45.064575 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.060864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:54297 (local address 127.18.211.131:33391)
0811 22:30:45.061002 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.061006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.061022 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:30:45.061082 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.061085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.061135 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.061272 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:45.061690 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.062255 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.063521 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.063916 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.064016 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.064060 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.064347 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.064444 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:30:45.065399 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.060749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34929 (local address 127.18.211.130:33585)
0811 22:30:45.061443 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.061457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.061582 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.061878 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.061881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.061897 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:45.062062 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.062069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.063200 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.063204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.064074 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.064084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.064995 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.064999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.065082 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.065102 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.065165 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.065276 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":311,"thread_start_us":166,"threads_started":1}
I20260811 22:30:45.065316 23107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6838fdac24c93b0a7fb92545deb66" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674" is_pre_election: true
I20260811 22:30:45.065436 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.060837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:33585 (local address 127.18.211.129:34929)
0811 22:30:45.060981 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.060986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.061575 (+ 589us) server_negotiation.cc:407] Connection header received
0811 22:30:45.061731 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.061734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.061777 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.061868 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:45.062615 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.063084 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.064212 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.064828 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.064961 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.065011 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.065241 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.065321 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:30:45.065711 23107 raft_consensus.cc:2454] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 0.
I20260811 22:30:45.066315 22735 leader_election.cc:302] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f44cd316376c45ceb802c8828663b674, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:45.066344 22599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6838fdac24c93b0a7fb92545deb66" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d646afff3dd40a894fbdddf4da617a0" is_pre_election: true
I20260811 22:30:45.066618 23254 raft_consensus.cc:2781] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:45.066709 23254 raft_consensus.cc:482] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:45.066681 22599 raft_consensus.cc:2454] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 0.
I20260811 22:30:45.066767 23254 raft_consensus.cc:3037] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.067939 23254 raft_consensus.cc:504] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.068378 23254 leader_election.cc:288] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 election: Requested vote from peers 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:45.068765 22599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6838fdac24c93b0a7fb92545deb66" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d646afff3dd40a894fbdddf4da617a0"
I20260811 22:30:45.068773 23107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f6838fdac24c93b0a7fb92545deb66" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674"
I20260811 22:30:45.068909 22599 raft_consensus.cc:3037] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.068929 23107 raft_consensus.cc:3037] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.070245 22599 raft_consensus.cc:2454] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 1.
I20260811 22:30:45.070266 23107 raft_consensus.cc:2454] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 1.
I20260811 22:30:45.070574 22737 leader_election.cc:302] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:45.070768 23254 raft_consensus.cc:2781] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:45.071030 23254 raft_consensus.cc:686] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 1 LEADER]: Becoming Leader. State: Replica: f480fbc8cc734489b4790d07d613435c, State: Running, Role: LEADER
I20260811 22:30:45.071421 23254 consensus_queue.cc:237] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.075132 22334 catalog_manager.cc:5495] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c reported cstate change: term changed from 0 to 1, leader changed from <none> to f480fbc8cc734489b4790d07d613435c (127.18.211.130). New cstate: current_term: 1 leader_uuid: "f480fbc8cc734489b4790d07d613435c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:30:45 dist-test-slave-8hhm krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
W20260811 22:30:45.116094 23197 tablet.cc:2369] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:30:45.120769 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.111141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39385 (local address 127.0.0.1:38166)
0811 22:30:45.111495 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.111509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.111624 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.111893 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.111895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.112353 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:45.112456 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.112463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.113168 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.113170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.113798 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.113808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.114249 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.114252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.114330 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.114911 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:45.114923 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:45.116700 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:45.118998 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:45.119009 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:45.119017 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:45.119278 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:45.119585 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:45.119589 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:45.119591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:45.119685 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:45.120106 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:45.120116 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:45.120237 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.120360 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.120536 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":123,"threads_started":1}
I20260811 22:30:45.120767 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.111288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.18.211.190:39385)
0811 22:30:45.111477 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.111482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.111593 (+ 111us) server_negotiation.cc:407] Connection header received
0811 22:30:45.111724 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.111727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.111770 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.111845 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:45.112567 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.113067 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.113909 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.114113 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.114202 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.116907 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:45.116911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:45.116913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:45.116934 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:45.118849 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:45.119391 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:45.119396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:45.119398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:45.119484 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:45.119783 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:45.119786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:45.119788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:45.119983 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:45.120095 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.120419 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.120594 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":101}
W20260811 22:30:45.122139 22935 tablet.cc:2369] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:30:45.125360 22334 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:38166:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:30:45.126272 22334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 22:30:45.129447 22687 tablet.cc:2369] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:30:45.131904 22816 tablet_service.cc:1467] Processing CreateTablet for tablet e41de09d74994ecbb9004edb607459c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf24b39232c14d6d93b190c709e7d087]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:30:45.131875 22569 tablet_service.cc:1467] Processing CreateTablet for tablet e41de09d74994ecbb9004edb607459c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf24b39232c14d6d93b190c709e7d087]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:30:45.132638 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41de09d74994ecbb9004edb607459c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:45.132659 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41de09d74994ecbb9004edb607459c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:45.134644 23068 tablet_service.cc:1467] Processing CreateTablet for tablet e41de09d74994ecbb9004edb607459c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf24b39232c14d6d93b190c709e7d087]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:30:45.135398 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41de09d74994ecbb9004edb607459c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:45.135602 23245 tablet_bootstrap.cc:492] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: Bootstrap starting.
I20260811 22:30:45.135636 23239 tablet_bootstrap.cc:492] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: Bootstrap starting.
I20260811 22:30:45.136605 23239 tablet_bootstrap.cc:654] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:45.136621 23245 tablet_bootstrap.cc:654] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:45.138183 23245 tablet_bootstrap.cc:492] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: No bootstrap required, opened a new log
I20260811 22:30:45.138293 23245 ts_tablet_manager.cc:1397] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:30:45.138487 23239 tablet_bootstrap.cc:492] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: No bootstrap required, opened a new log
I20260811 22:30:45.138586 23239 ts_tablet_manager.cc:1397] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:30:45.138799 23245 raft_consensus.cc:348] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.138880 23238 tablet_bootstrap.cc:492] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: Bootstrap starting.
I20260811 22:30:45.138921 23245 raft_consensus.cc:374] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:45.138957 23245 raft_consensus.cc:729] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f480fbc8cc734489b4790d07d613435c, State: Initialized, Role: FOLLOWER
I20260811 22:30:45.139062 23239 raft_consensus.cc:348] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.139102 23245 consensus_queue.cc:260] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.139175 23239 raft_consensus.cc:374] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:45.139218 23239 raft_consensus.cc:729] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Initialized, Role: FOLLOWER
I20260811 22:30:45.139340 23239 consensus_queue.cc:260] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.139422 23245 ts_tablet_manager.cc:1428] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 22:30:45.139621 23239 ts_tablet_manager.cc:1428] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:30:45.139849 23253 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.139956 23238 tablet_bootstrap.cc:654] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:45.139909 23254 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.140017 23253 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:45.140059 23254 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:45.141487 23238 tablet_bootstrap.cc:492] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: No bootstrap required, opened a new log
I20260811 22:30:45.141602 23238 ts_tablet_manager.cc:1397] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:30:45.142076 23238 raft_consensus.cc:348] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.142204 23238 raft_consensus.cc:374] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:45.142251 23238 raft_consensus.cc:729] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44cd316376c45ceb802c8828663b674, State: Initialized, Role: FOLLOWER
I20260811 22:30:45.142385 23238 consensus_queue.cc:260] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.142686 23238 ts_tablet_manager.cc:1428] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:30:45.142707 23249 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.142875 23249 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:45.234966 23253 raft_consensus.cc:482] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:45.235129 23253 raft_consensus.cc:504] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.236414 23253 leader_election.cc:288] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:45.241197 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.236807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:42333 (local address 127.18.211.131:33391)
0811 22:30:45.236957 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.236963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.237177 (+ 214us) server_negotiation.cc:407] Connection header received
0811 22:30:45.237366 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.237370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.237423 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.237510 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:45.237978 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.238751 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.240008 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.240583 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.240680 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.240722 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.240976 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.241030 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:30:45.241304 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.236668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:33391 (local address 127.18.211.129:42333)
0811 22:30:45.237066 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.237079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.237197 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.237536 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.237539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.237556 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:45.237743 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.237752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.238871 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.238874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.239876 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.239885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.240727 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.240731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.240837 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.240866 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.240948 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.241017 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":3,"thread_start_us":156,"threads_started":1}
I20260811 22:30:45.241670 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.236750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:45933 (local address 127.18.211.130:43791)
0811 22:30:45.236896 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.236901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.237395 (+ 494us) server_negotiation.cc:407] Connection header received
0811 22:30:45.237506 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.237509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.237564 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.237643 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:30:45.238615 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.239312 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.240471 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.241044 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.241157 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.241203 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.241492 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.241568 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:30:45.241684 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.236686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:43791 (local address 127.18.211.129:45933)
0811 22:30:45.237266 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.237282 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.237395 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.237677 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.237679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.237694 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:30:45.238484 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.238495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.239433 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.239436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.240303 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.240313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.241168 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.241172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.241316 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.241341 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.241468 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.241536 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":22,"thread_start_us":139,"threads_started":1}
I20260811 22:30:45.242125 23107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41de09d74994ecbb9004edb607459c0" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674" is_pre_election: true
I20260811 22:30:45.242302 23107 raft_consensus.cc:2454] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 0.
I20260811 22:30:45.242347 22850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41de09d74994ecbb9004edb607459c0" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f480fbc8cc734489b4790d07d613435c" is_pre_election: true
I20260811 22:30:45.242605 22850 raft_consensus.cc:2454] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 0.
I20260811 22:30:45.242707 22483 leader_election.cc:302] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f44cd316376c45ceb802c8828663b674; no voters:
I20260811 22:30:45.243057 23253 raft_consensus.cc:2781] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:45.243173 23253 raft_consensus.cc:482] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:45.243237 23253 raft_consensus.cc:3037] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.244272 23253 raft_consensus.cc:504] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.244650 23253 leader_election.cc:288] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 election: Requested vote from peers f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:45.245025 23107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41de09d74994ecbb9004edb607459c0" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674"
I20260811 22:30:45.245134 23107 raft_consensus.cc:3037] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.245085 22850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41de09d74994ecbb9004edb607459c0" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f480fbc8cc734489b4790d07d613435c"
I20260811 22:30:45.245266 22850 raft_consensus.cc:3037] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.246172 23107 raft_consensus.cc:2454] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 1.
I20260811 22:30:45.246310 22850 raft_consensus.cc:2454] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 1.
I20260811 22:30:45.246452 22483 leader_election.cc:302] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f44cd316376c45ceb802c8828663b674; no voters:
I20260811 22:30:45.246740 23253 raft_consensus.cc:2781] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:45.246937 23253 raft_consensus.cc:686] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [term 1 LEADER]: Becoming Leader. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Running, Role: LEADER
I20260811 22:30:45.247351 23253 consensus_queue.cc:237] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.249914 23280 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: TxnStatusTablet state changed. Reason: New leader 3d646afff3dd40a894fbdddf4da617a0. Latest consensus state: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.250036 23280 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:30:45.250313 23285 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:30:45.250558 23285 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:30:45.251194 22333 catalog_manager.cc:5495] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260811 22:30:45.265143 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.260886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34929 (local address 127.0.0.1:56532)
0811 22:30:45.261030 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.261048 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.261162 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.261593 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.261595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.261612 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:30:45.261709 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.261713 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.262758 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.262761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.263375 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.263384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.263954 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.263956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.264022 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.264070 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.264879 (+ 809us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.264892 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.264945 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.265020 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 22:30:45.265161 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.260992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56532 (local address 127.18.211.129:34929)
0811 22:30:45.261124 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.261128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.261145 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:30:45.261389 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.261393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.261466 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.261575 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:45.261817 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.262593 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.263499 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.263839 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.263949 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.264217 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.264781 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.264903 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.264962 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.265019 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:30:45.270689 22850 raft_consensus.cc:1261] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Refusing update from remote peer 3d646afff3dd40a894fbdddf4da617a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:45.270709 23107 raft_consensus.cc:1261] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Refusing update from remote peer 3d646afff3dd40a894fbdddf4da617a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:45.271387 23280 consensus_queue.cc:1035] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:45.271617 23253 consensus_queue.cc:1035] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:45.275377 23254 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: TxnStatusTablet state changed. Reason: New leader 3d646afff3dd40a894fbdddf4da617a0. Latest consensus state: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.275511 23254 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:45.277905 23249 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: TxnStatusTablet state changed. Reason: New leader 3d646afff3dd40a894fbdddf4da617a0. Latest consensus state: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.278046 23249 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:45.280791 23254 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.280805 23253 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.280920 23253 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:30:45.280923 23254 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P f480fbc8cc734489b4790d07d613435c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:45.281023 23253 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.281088 23253 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P 3d646afff3dd40a894fbdddf4da617a0: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:30:45.281411 23290 mvcc.cc:205] Tried to move back new op lower bound from 7317452575820988416 to 7317452575736004608. Current Snapshot: MvccSnapshot[applied={T|T < 7317452575820988416}]
I20260811 22:30:45.282428 23291 mvcc.cc:205] Tried to move back new op lower bound from 7317452575820988416 to 7317452575736004608. Current Snapshot: MvccSnapshot[applied={T|T < 7317452575820988416}]
I20260811 22:30:45.282895 23249 tablet_replica.cc:439] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } } }
I20260811 22:30:45.283030 23249 tablet_replica.cc:442] T e41de09d74994ecbb9004edb607459c0 P f44cd316376c45ceb802c8828663b674: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:30:45.283524 23293 mvcc.cc:205] Tried to move back new op lower bound from 7317452575820988416 to 7317452575736004608. Current Snapshot: MvccSnapshot[applied={T|T < 7317452575820988416}]
I20260811 22:30:45.293457 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.289755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.18.211.130:43791)
0811 22:30:45.289863 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.289869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.289904 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:30:45.290108 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.290111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.290174 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.290287 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:45.290543 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.291051 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.291940 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.292174 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.292286 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.292580 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.293022 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.293158 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.293250 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.293308 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:30:45.305518 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.301653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:35633 (local address 127.18.211.129:34929)
0811 22:30:45.301809 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.301814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.301968 (+ 154us) server_negotiation.cc:407] Connection header received
0811 22:30:45.302078 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.302081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.302135 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.302246 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:45.302557 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.303330 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.304192 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.304549 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.304680 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.304993 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.305081 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.305154 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.305315 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.305371 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:30:45.305570 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.301538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34929 (local address 127.18.211.130:35633)
0811 22:30:45.301877 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.301890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.301980 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.302259 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.302263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.302271 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:30:45.302445 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.302451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.303442 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.303444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.304048 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.304057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.304705 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.304708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.304781 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.304827 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.305168 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.305174 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.305300 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.305405 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":145,"threads_started":1}
I20260811 22:30:45.312878 23107 raft_consensus.cc:1261] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Refusing update from remote peer f480fbc8cc734489b4790d07d613435c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:45.313081 22599 raft_consensus.cc:1261] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Refusing update from remote peer f480fbc8cc734489b4790d07d613435c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:45.313642 23254 consensus_queue.cc:1035] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:45.313910 23258 consensus_queue.cc:1035] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:45.339156 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.334890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:38791 (local address 127.18.211.130:43791)
0811 22:30:45.335026 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.335031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.335289 (+ 258us) server_negotiation.cc:407] Connection header received
0811 22:30:45.335415 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.335417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.335459 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.335522 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:45.335848 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.336636 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.337965 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.338209 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.338293 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.338674 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.338743 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.338795 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.338933 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.339017 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:30:45.339169 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.334811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:43791 (local address 127.18.211.129:38791)
0811 22:30:45.335200 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.335216 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.335327 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.335563 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.335566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.335573 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:30:45.335713 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.335720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.336794 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.336797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.337841 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.337853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.338363 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.338367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.338474 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.338537 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.338839 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.338845 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.338916 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.338994 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":112,"threads_started":1}
I20260811 22:30:45.370260 22333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:45.370486 22333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:45.373769 22333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Sending DeleteTablet for 3 replicas of tablet 49f6838fdac24c93b0a7fb92545deb66
I20260811 22:30:45.374805 23068 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f6838fdac24c93b0a7fb92545deb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:45 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56186
I20260811 22:30:45.374908 22816 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f6838fdac24c93b0a7fb92545deb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:45 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55480
I20260811 22:30:45.375315 23330 tablet_replica.cc:330] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: stopping tablet replica
I20260811 22:30:45.375633 23330 raft_consensus.cc:2229] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:45.375579 22569 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f6838fdac24c93b0a7fb92545deb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:45 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56528
I20260811 22:30:45.375874 23330 raft_consensus.cc:2258] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:45.377144 22333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:30:45.377476 23330 ts_tablet_manager.cc:1905] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:30:45.377650 22333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:45.379132 23333 tablet_replica.cc:330] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: stopping tablet replica
I20260811 22:30:45.379417 23331 tablet_replica.cc:330] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: stopping tablet replica
I20260811 22:30:45.380684 23331 raft_consensus.cc:2229] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:30:45.381560 23333 raft_consensus.cc:2229] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:45.382704 23068 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b (DEFAULT_TABLE table=test-table [id=31633a95dc67487e9e0306c4efcf1319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:45.383035 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:45.383292 23333 raft_consensus.cc:2258] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:45.383634 23331 raft_consensus.cc:2258] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:45.384882 23330 ts_tablet_manager.cc:1918] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:45.384989 23330 log.cc:1196] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/49f6838fdac24c93b0a7fb92545deb66
I20260811 22:30:45.385370 23330 ts_tablet_manager.cc:1939] T 49f6838fdac24c93b0a7fb92545deb66 P f44cd316376c45ceb802c8828663b674: Deleting consensus metadata
I20260811 22:30:45.386814 23333 ts_tablet_manager.cc:1905] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:45.386868 23331 ts_tablet_manager.cc:1905] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:45.387044 22816 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b (DEFAULT_TABLE table=test-table [id=31633a95dc67487e9e0306c4efcf1319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:45.387324 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:45.386826 22569 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b (DEFAULT_TABLE table=test-table [id=31633a95dc67487e9e0306c4efcf1319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:45.389605 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:45.389935 23331 ts_tablet_manager.cc:1918] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:45.390043 23331 log.cc:1196] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/49f6838fdac24c93b0a7fb92545deb66
I20260811 22:30:45.390367 23331 ts_tablet_manager.cc:1939] T 49f6838fdac24c93b0a7fb92545deb66 P f480fbc8cc734489b4790d07d613435c: Deleting consensus metadata
I20260811 22:30:45.392160 23239 tablet_bootstrap.cc:492] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: Bootstrap starting.
I20260811 22:30:45.392257 23245 tablet_bootstrap.cc:492] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: Bootstrap starting.
I20260811 22:30:45.392491 22315 catalog_manager.cc:4841] TS f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791): tablet 49f6838fdac24c93b0a7fb92545deb66 (table test-table [id=731e334414694a09b4455093a89333df]) successfully deleted
I20260811 22:30:45.393234 23245 tablet_bootstrap.cc:654] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:45.393247 23239 tablet_bootstrap.cc:654] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:45.393875 23238 tablet_bootstrap.cc:492] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: Bootstrap starting.
I20260811 22:30:45.394778 23238 tablet_bootstrap.cc:654] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:45.397087 23238 tablet_bootstrap.cc:492] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: No bootstrap required, opened a new log
I20260811 22:30:45.397192 23238 ts_tablet_manager.cc:1397] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:30:45.397429 23333 ts_tablet_manager.cc:1918] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:45.397621 23333 log.cc:1196] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/49f6838fdac24c93b0a7fb92545deb66
I20260811 22:30:45.397644 23238 raft_consensus.cc:348] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.397770 23238 raft_consensus.cc:374] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:45.397806 23238 raft_consensus.cc:729] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44cd316376c45ceb802c8828663b674, State: Initialized, Role: FOLLOWER
I20260811 22:30:45.397940 23238 consensus_queue.cc:260] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.398222 23238 ts_tablet_manager.cc:1428] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:45.398381 23333 ts_tablet_manager.cc:1939] T 49f6838fdac24c93b0a7fb92545deb66 P 3d646afff3dd40a894fbdddf4da617a0: Deleting consensus metadata
I20260811 22:30:45.399780 22315 catalog_manager.cc:4841] TS 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929): tablet 49f6838fdac24c93b0a7fb92545deb66 (table test-table [id=731e334414694a09b4455093a89333df]) successfully deleted
I20260811 22:30:45.401916 23239 tablet_bootstrap.cc:492] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: No bootstrap required, opened a new log
I20260811 22:30:45.402006 23239 ts_tablet_manager.cc:1397] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20260811 22:30:45.402482 23239 raft_consensus.cc:348] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.402614 23239 raft_consensus.cc:374] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:45.402644 23239 raft_consensus.cc:729] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Initialized, Role: FOLLOWER
I20260811 22:30:45.402761 23239 consensus_queue.cc:260] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.402997 23239 ts_tablet_manager.cc:1428] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:45.403668 22311 catalog_manager.cc:4841] TS f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391): tablet 49f6838fdac24c93b0a7fb92545deb66 (table test-table [id=731e334414694a09b4455093a89333df]) successfully deleted
I20260811 22:30:45.405480 23245 tablet_bootstrap.cc:492] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: No bootstrap required, opened a new log
I20260811 22:30:45.405581 23245 ts_tablet_manager.cc:1397] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260811 22:30:45.406011 23245 raft_consensus.cc:348] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.406134 23245 raft_consensus.cc:374] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:45.406167 23245 raft_consensus.cc:729] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f480fbc8cc734489b4790d07d613435c, State: Initialized, Role: FOLLOWER
I20260811 22:30:45.406301 23245 consensus_queue.cc:260] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.406551 23245 ts_tablet_manager.cc:1428] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:45.485780 23315 raft_consensus.cc:482] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:45.485963 23315 raft_consensus.cc:504] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.486442 23315 leader_election.cc:288] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:45.486994 23106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8ebdaa4a3d4684b717b3ef9a62e62b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674" is_pre_election: true
I20260811 22:30:45.486994 22850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8ebdaa4a3d4684b717b3ef9a62e62b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f480fbc8cc734489b4790d07d613435c" is_pre_election: true
I20260811 22:30:45.487179 22850 raft_consensus.cc:2454] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 0.
I20260811 22:30:45.487192 23106 raft_consensus.cc:2454] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 0.
I20260811 22:30:45.487514 22489 leader_election.cc:302] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:45.487703 23315 raft_consensus.cc:2781] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:45.487761 23315 raft_consensus.cc:482] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:45.487798 23315 raft_consensus.cc:3037] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.489251 23315 raft_consensus.cc:504] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.489634 23315 leader_election.cc:288] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 election: Requested vote from peers f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:45.490064 22850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8ebdaa4a3d4684b717b3ef9a62e62b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f480fbc8cc734489b4790d07d613435c"
I20260811 22:30:45.490234 22850 raft_consensus.cc:3037] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.491178 22850 raft_consensus.cc:2454] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 1.
I20260811 22:30:45.491510 23106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8ebdaa4a3d4684b717b3ef9a62e62b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674"
I20260811 22:30:45.491587 22489 leader_election.cc:302] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:45.491650 23106 raft_consensus.cc:3037] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:45.491868 23315 raft_consensus.cc:2781] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:45.491964 23315 raft_consensus.cc:686] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 LEADER]: Becoming Leader. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Running, Role: LEADER
I20260811 22:30:45.492123 23315 consensus_queue.cc:237] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:45.492930 23106 raft_consensus.cc:2454] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 1.
I20260811 22:30:45.494663 22332 catalog_manager.cc:5495] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260811 22:30:45.531625 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.528110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.18.211.129:34929)
0811 22:30:45.528219 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.528223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.528235 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:30:45.528370 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.528372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.528415 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.528475 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:45.528795 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.529618 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.530484 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.530804 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.530920 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.531151 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.531239 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.531323 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.531406 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.531465 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 22:30:45.542198 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.537799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34929 (local address 127.18.211.129:58897)
0811 22:30:45.537967 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:45.537980 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:45.538047 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:45.539559 (+ 1512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:45.539561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:45.539567 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:30:45.539700 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.539705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.540406 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.540410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.540977 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:45.540985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.541490 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.541493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:45.541572 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.541593 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.541893 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.541902 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:30:45.541992 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:30:45.542069 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":25}
I20260811 22:30:45.542198 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:45.539261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:58897 (local address 127.18.211.129:34929)
0811 22:30:45.539377 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:45.539380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:45.539391 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:30:45.539422 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:45.539424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:45.539458 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:45.539551 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:45.539803 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.540293 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.541099 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:45.541377 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:45.541491 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:45.541729 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:45.541806 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:45.541889 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:45.542003 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 22:30:45.542044 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":55}
I20260811 22:30:45.548401 22850 raft_consensus.cc:1261] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Refusing update from remote peer 3d646afff3dd40a894fbdddf4da617a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:45.548493 23106 raft_consensus.cc:1261] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Refusing update from remote peer 3d646afff3dd40a894fbdddf4da617a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:45.548897 23280 consensus_queue.cc:1035] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:45.549059 23315 consensus_queue.cc:1035] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:30:45.567842 23290 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:30:47.142072 22333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:47.142247 22333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:47.145041 22333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Sending DeleteTablet for 3 replicas of tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b
I20260811 22:30:47.145758 22569 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:47 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56528
I20260811 22:30:47.145834 23068 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:47 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56186
I20260811 22:30:47.145856 22816 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:47 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55480
I20260811 22:30:47.146180 23392 tablet_replica.cc:330] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: stopping tablet replica
I20260811 22:30:47.146235 23397 tablet_replica.cc:330] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: stopping tablet replica
I20260811 22:30:47.146353 23392 raft_consensus.cc:2229] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:30:47.146387 23397 raft_consensus.cc:2229] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:47.146540 23397 raft_consensus.cc:2258] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:47.146719 23394 tablet_replica.cc:330] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: stopping tablet replica
I20260811 22:30:47.146744 23392 raft_consensus.cc:2258] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:47.146927 23394 raft_consensus.cc:2229] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:47.147104 23394 raft_consensus.cc:2258] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:47.147400 23397 ts_tablet_manager.cc:1905] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:47.147890 23394 ts_tablet_manager.cc:1905] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:47.150434 23392 ts_tablet_manager.cc:1905] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:47.150790 23397 ts_tablet_manager.cc:1918] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:30:47.150897 23397 log.cc:1196] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/2f8ebdaa4a3d4684b717b3ef9a62e62b
I20260811 22:30:47.151104 23394 ts_tablet_manager.cc:1918] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:30:47.151190 23394 log.cc:1196] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/2f8ebdaa4a3d4684b717b3ef9a62e62b
I20260811 22:30:47.151201 23397 ts_tablet_manager.cc:1939] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f44cd316376c45ceb802c8828663b674: Deleting consensus metadata
I20260811 22:30:47.151463 23394 ts_tablet_manager.cc:1939] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P f480fbc8cc734489b4790d07d613435c: Deleting consensus metadata
I20260811 22:30:47.152101 22311 catalog_manager.cc:4841] TS f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391): tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b (table test-table [id=31633a95dc67487e9e0306c4efcf1319]) successfully deleted
I20260811 22:30:47.152254 22315 catalog_manager.cc:4841] TS f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791): tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b (table test-table [id=31633a95dc67487e9e0306c4efcf1319]) successfully deleted
I20260811 22:30:47.153409 23392 ts_tablet_manager.cc:1918] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:30:47.153486 23392 log.cc:1196] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/2f8ebdaa4a3d4684b717b3ef9a62e62b
I20260811 22:30:47.153815 23392 ts_tablet_manager.cc:1939] T 2f8ebdaa4a3d4684b717b3ef9a62e62b P 3d646afff3dd40a894fbdddf4da617a0: Deleting consensus metadata
I20260811 22:30:47.154642 22315 catalog_manager.cc:4841] TS 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929): tablet 2f8ebdaa4a3d4684b717b3ef9a62e62b (table test-table [id=31633a95dc67487e9e0306c4efcf1319]) successfully deleted
Aug 11 22:30:47 dist-test-slave-8hhm krb5kdc[22259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:30:47 dist-test-slave-8hhm krb5kdc[22259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:47.179563 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:47.169653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.18.211.190:39385)
0811 22:30:47.169985 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:47.169993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:47.170390 (+ 397us) server_negotiation.cc:407] Connection header received
0811 22:30:47.170484 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:47.170487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:47.170544 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:47.170654 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:47.171559 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:47.172141 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:47.172990 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:47.173344 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:47.173430 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:47.176319 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:47.176322 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:47.176323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:47.176353 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:47.177947 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:47.178406 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:47.178409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:47.178410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:47.178497 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:47.178752 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:47.178753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:47.178755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:47.178945 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:47.179050 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:47.179310 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 22:30:47.179422 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":99,"threads_started":1}
I20260811 22:30:47.182550 22333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:30:47.183130 22333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:47.188536 22816 tablet_service.cc:1467] Processing CreateTablet for tablet ada156a5be624523b333fc21183f983b (DEFAULT_TABLE table=test-table [id=b01f171d401744d6afbc9a1b6a49bb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:47.188530 22569 tablet_service.cc:1467] Processing CreateTablet for tablet ada156a5be624523b333fc21183f983b (DEFAULT_TABLE table=test-table [id=b01f171d401744d6afbc9a1b6a49bb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:47.188764 23068 tablet_service.cc:1467] Processing CreateTablet for tablet ada156a5be624523b333fc21183f983b (DEFAULT_TABLE table=test-table [id=b01f171d401744d6afbc9a1b6a49bb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:47.188858 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada156a5be624523b333fc21183f983b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:47.188856 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada156a5be624523b333fc21183f983b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:47.189069 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada156a5be624523b333fc21183f983b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:47.191202 23413 tablet_bootstrap.cc:492] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: Bootstrap starting.
I20260811 22:30:47.191649 23416 tablet_bootstrap.cc:492] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: Bootstrap starting.
I20260811 22:30:47.192617 23416 tablet_bootstrap.cc:654] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:47.192759 23413 tablet_bootstrap.cc:654] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:47.193298 23417 tablet_bootstrap.cc:492] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: Bootstrap starting.
I20260811 22:30:47.194046 23413 tablet_bootstrap.cc:492] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: No bootstrap required, opened a new log
I20260811 22:30:47.194124 23417 tablet_bootstrap.cc:654] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:47.194165 23413 ts_tablet_manager.cc:1397] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:30:47.194664 23413 raft_consensus.cc:348] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.194810 23413 raft_consensus.cc:374] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:47.194851 23413 raft_consensus.cc:729] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44cd316376c45ceb802c8828663b674, State: Initialized, Role: FOLLOWER
I20260811 22:30:47.195057 23416 tablet_bootstrap.cc:492] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: No bootstrap required, opened a new log
I20260811 22:30:47.195026 23413 consensus_queue.cc:260] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.195182 23416 ts_tablet_manager.cc:1397] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:30:47.195446 23413 ts_tablet_manager.cc:1428] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 22:30:47.195617 23416 raft_consensus.cc:348] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.195735 23416 raft_consensus.cc:374] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:47.195775 23416 raft_consensus.cc:729] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Initialized, Role: FOLLOWER
I20260811 22:30:47.195910 23416 consensus_queue.cc:260] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.196216 23416 ts_tablet_manager.cc:1428] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:30:47.196756 23417 tablet_bootstrap.cc:492] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: No bootstrap required, opened a new log
I20260811 22:30:47.196851 23417 ts_tablet_manager.cc:1397] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 22:30:47.197304 23417 raft_consensus.cc:348] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.197412 23417 raft_consensus.cc:374] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:47.197458 23417 raft_consensus.cc:729] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f480fbc8cc734489b4790d07d613435c, State: Initialized, Role: FOLLOWER
I20260811 22:30:47.197587 23417 consensus_queue.cc:260] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.197997 23417 ts_tablet_manager.cc:1428] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:47.230088 23424 raft_consensus.cc:482] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:47.230278 23424 raft_consensus.cc:504] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.230772 23424 leader_election.cc:288] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:47.231298 22599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada156a5be624523b333fc21183f983b" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d646afff3dd40a894fbdddf4da617a0" is_pre_election: true
I20260811 22:30:47.231348 23106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada156a5be624523b333fc21183f983b" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674" is_pre_election: true
I20260811 22:30:47.231478 23106 raft_consensus.cc:2454] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 0.
I20260811 22:30:47.231479 22599 raft_consensus.cc:2454] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 0.
I20260811 22:30:47.231881 22737 leader_election.cc:302] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f44cd316376c45ceb802c8828663b674, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:47.232146 23424 raft_consensus.cc:2781] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:47.232211 23424 raft_consensus.cc:482] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:47.232240 23424 raft_consensus.cc:3037] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:47.233402 23424 raft_consensus.cc:504] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.233760 23424 leader_election.cc:288] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 election: Requested vote from peers 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:47.234201 22599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada156a5be624523b333fc21183f983b" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d646afff3dd40a894fbdddf4da617a0"
I20260811 22:30:47.234201 23106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada156a5be624523b333fc21183f983b" candidate_uuid: "f480fbc8cc734489b4790d07d613435c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674"
I20260811 22:30:47.234347 23106 raft_consensus.cc:3037] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:47.234346 22599 raft_consensus.cc:3037] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:47.235561 23106 raft_consensus.cc:2454] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 1.
I20260811 22:30:47.235560 22599 raft_consensus.cc:2454] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f480fbc8cc734489b4790d07d613435c in term 1.
I20260811 22:30:47.235911 22735 leader_election.cc:302] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f44cd316376c45ceb802c8828663b674, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:47.236157 23424 raft_consensus.cc:2781] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:47.236446 23424 raft_consensus.cc:686] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 1 LEADER]: Becoming Leader. State: Replica: f480fbc8cc734489b4790d07d613435c, State: Running, Role: LEADER
I20260811 22:30:47.236644 23424 consensus_queue.cc:237] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.238581 22333 catalog_manager.cc:5495] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c reported cstate change: term changed from 0 to 1, leader changed from <none> to f480fbc8cc734489b4790d07d613435c (127.18.211.130). New cstate: current_term: 1 leader_uuid: "f480fbc8cc734489b4790d07d613435c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260811 22:30:47.255471 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:47.251967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.18.211.130:43791)
0811 22:30:47.252285 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:47.252295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:47.252310 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:30:47.252359 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:47.252361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:47.252403 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:47.252498 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:47.252792 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:47.253395 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:47.254310 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:47.254571 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:47.254692 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:47.254969 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:47.255059 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:47.255147 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:47.255247 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 22:30:47.255301 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":106,"threads_started":1}
I20260811 22:30:47.262425 23106 raft_consensus.cc:1261] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Refusing update from remote peer f480fbc8cc734489b4790d07d613435c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:47.262462 22599 raft_consensus.cc:1261] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Refusing update from remote peer f480fbc8cc734489b4790d07d613435c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:47.262943 23424 consensus_queue.cc:1035] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:47.263156 23425 consensus_queue.cc:1035] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:47.299643 22334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:47.299801 22334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:47.302155 22334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Sending DeleteTablet for 3 replicas of tablet ada156a5be624523b333fc21183f983b
I20260811 22:30:47.302811 22569 tablet_service.cc:1514] Processing DeleteTablet for tablet ada156a5be624523b333fc21183f983b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:47 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56528
I20260811 22:30:47.302894 23068 tablet_service.cc:1514] Processing DeleteTablet for tablet ada156a5be624523b333fc21183f983b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:47 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56186
I20260811 22:30:47.303037 23392 tablet_replica.cc:330] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: stopping tablet replica
I20260811 22:30:47.303097 22816 tablet_service.cc:1514] Processing DeleteTablet for tablet ada156a5be624523b333fc21183f983b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:47 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55480
I20260811 22:30:47.303117 23397 tablet_replica.cc:330] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: stopping tablet replica
I20260811 22:30:47.303174 23392 raft_consensus.cc:2229] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:47.303259 23394 tablet_replica.cc:330] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: stopping tablet replica
I20260811 22:30:47.303253 23397 raft_consensus.cc:2229] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:47.303299 23392 raft_consensus.cc:2258] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:47.303369 23394 raft_consensus.cc:2229] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:30:47.303387 23397 raft_consensus.cc:2258] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:47.303555 23394 raft_consensus.cc:2258] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:47.304010 22334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:30:47.304232 23397 ts_tablet_manager.cc:1905] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:47.304291 23394 ts_tablet_manager.cc:1905] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:47.304314 23392 ts_tablet_manager.cc:1905] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:30:47.304453 22334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:30:47.306901 23397 ts_tablet_manager.cc:1918] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:47.306901 23394 ts_tablet_manager.cc:1918] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:47.306986 23394 log.cc:1196] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/ada156a5be624523b333fc21183f983b
I20260811 22:30:47.306986 23397 log.cc:1196] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/ada156a5be624523b333fc21183f983b
I20260811 22:30:47.307260 23394 ts_tablet_manager.cc:1939] T ada156a5be624523b333fc21183f983b P f480fbc8cc734489b4790d07d613435c: Deleting consensus metadata
I20260811 22:30:47.307260 23397 ts_tablet_manager.cc:1939] T ada156a5be624523b333fc21183f983b P f44cd316376c45ceb802c8828663b674: Deleting consensus metadata
I20260811 22:30:47.307480 23392 ts_tablet_manager.cc:1918] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:30:47.307569 23392 log.cc:1196] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/ada156a5be624523b333fc21183f983b
I20260811 22:30:47.307837 23392 ts_tablet_manager.cc:1939] T ada156a5be624523b333fc21183f983b P 3d646afff3dd40a894fbdddf4da617a0: Deleting consensus metadata
I20260811 22:30:47.307955 22315 catalog_manager.cc:4841] TS f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791): tablet ada156a5be624523b333fc21183f983b (table test-table [id=b01f171d401744d6afbc9a1b6a49bb40]) successfully deleted
I20260811 22:30:47.307955 22311 catalog_manager.cc:4841] TS f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391): tablet ada156a5be624523b333fc21183f983b (table test-table [id=b01f171d401744d6afbc9a1b6a49bb40]) successfully deleted
I20260811 22:30:47.308598 22315 catalog_manager.cc:4841] TS 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929): tablet ada156a5be624523b333fc21183f983b (table test-table [id=b01f171d401744d6afbc9a1b6a49bb40]) successfully deleted
I20260811 22:30:47.309584 22569 tablet_service.cc:1467] Processing CreateTablet for tablet 042e387083a44539be0b1d4a9fd01c3b (DEFAULT_TABLE table=test-table [id=cf738a90e0af432193e746fff7ec4c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:47.309674 22816 tablet_service.cc:1467] Processing CreateTablet for tablet 042e387083a44539be0b1d4a9fd01c3b (DEFAULT_TABLE table=test-table [id=cf738a90e0af432193e746fff7ec4c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:47.309878 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 042e387083a44539be0b1d4a9fd01c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:47.309860 23068 tablet_service.cc:1467] Processing CreateTablet for tablet 042e387083a44539be0b1d4a9fd01c3b (DEFAULT_TABLE table=test-table [id=cf738a90e0af432193e746fff7ec4c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:30:47.309978 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 042e387083a44539be0b1d4a9fd01c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:47.310096 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 042e387083a44539be0b1d4a9fd01c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:47.311779 23417 tablet_bootstrap.cc:492] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: Bootstrap starting.
I20260811 22:30:47.312310 23413 tablet_bootstrap.cc:492] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: Bootstrap starting.
I20260811 22:30:47.312686 23417 tablet_bootstrap.cc:654] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:47.313117 23413 tablet_bootstrap.cc:654] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:47.313195 23416 tablet_bootstrap.cc:492] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0: Bootstrap starting.
I20260811 22:30:47.313820 23417 tablet_bootstrap.cc:492] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: No bootstrap required, opened a new log
I20260811 22:30:47.313902 23417 ts_tablet_manager.cc:1397] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:30:47.313975 23416 tablet_bootstrap.cc:654] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:47.314405 23413 tablet_bootstrap.cc:492] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: No bootstrap required, opened a new log
I20260811 22:30:47.314363 23417 raft_consensus.cc:348] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.314479 23413 ts_tablet_manager.cc:1397] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:30:47.314481 23417 raft_consensus.cc:374] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:47.314517 23417 raft_consensus.cc:729] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f480fbc8cc734489b4790d07d613435c, State: Initialized, Role: FOLLOWER
I20260811 22:30:47.314651 23417 consensus_queue.cc:260] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.314924 23417 ts_tablet_manager.cc:1428] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:47.314924 23413 raft_consensus.cc:348] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.315042 23413 raft_consensus.cc:374] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:47.315084 23413 raft_consensus.cc:729] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44cd316376c45ceb802c8828663b674, State: Initialized, Role: FOLLOWER
I20260811 22:30:47.315047 23416 tablet_bootstrap.cc:492] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0: No bootstrap required, opened a new log
I20260811 22:30:47.315182 23416 ts_tablet_manager.cc:1397] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:30:47.315208 23413 consensus_queue.cc:260] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.315490 23413 ts_tablet_manager.cc:1428] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:30:47.315629 23416 raft_consensus.cc:348] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.315734 23416 raft_consensus.cc:374] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:47.315773 23416 raft_consensus.cc:729] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Initialized, Role: FOLLOWER
I20260811 22:30:47.315893 23416 consensus_queue.cc:260] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.316186 23416 ts_tablet_manager.cc:1428] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:30:47.461184 23347 raft_consensus.cc:482] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:30:47.461411 23347 raft_consensus.cc:504] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.461935 23347 leader_election.cc:288] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:47.462534 22850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042e387083a44539be0b1d4a9fd01c3b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f480fbc8cc734489b4790d07d613435c" is_pre_election: true
I20260811 22:30:47.462534 23107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042e387083a44539be0b1d4a9fd01c3b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674" is_pre_election: true
I20260811 22:30:47.462687 22850 raft_consensus.cc:2454] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 0.
I20260811 22:30:47.462747 23107 raft_consensus.cc:2454] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 0.
I20260811 22:30:47.463146 22489 leader_election.cc:302] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:47.463349 23347 raft_consensus.cc:2781] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:30:47.463411 23347 raft_consensus.cc:482] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:30:47.463439 23347 raft_consensus.cc:3037] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:47.464529 23347 raft_consensus.cc:504] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.464910 23347 leader_election.cc:288] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 election: Requested vote from peers f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791), f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391)
I20260811 22:30:47.465349 22850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042e387083a44539be0b1d4a9fd01c3b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f480fbc8cc734489b4790d07d613435c"
I20260811 22:30:47.465365 23107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042e387083a44539be0b1d4a9fd01c3b" candidate_uuid: "3d646afff3dd40a894fbdddf4da617a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44cd316376c45ceb802c8828663b674"
I20260811 22:30:47.465503 22850 raft_consensus.cc:3037] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:47.465508 23107 raft_consensus.cc:3037] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:47.466502 23107 raft_consensus.cc:2454] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 1.
I20260811 22:30:47.466513 22850 raft_consensus.cc:2454] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d646afff3dd40a894fbdddf4da617a0 in term 1.
I20260811 22:30:47.466861 22489 leader_election.cc:302] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d646afff3dd40a894fbdddf4da617a0, f480fbc8cc734489b4790d07d613435c; no voters:
I20260811 22:30:47.467077 23347 raft_consensus.cc:2781] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:47.467180 23347 raft_consensus.cc:686] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 LEADER]: Becoming Leader. State: Replica: 3d646afff3dd40a894fbdddf4da617a0, State: Running, Role: LEADER
I20260811 22:30:47.467365 23347 consensus_queue.cc:237] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } }
I20260811 22:30:47.469336 22334 catalog_manager.cc:5495] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "3d646afff3dd40a894fbdddf4da617a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d646afff3dd40a894fbdddf4da617a0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260811 22:30:47.518591 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:47.515327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.18.211.129:34929)
0811 22:30:47.515608 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:47.515614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:47.515626 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:30:47.515665 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:47.515667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:47.515719 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:47.515793 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:47.516157 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:47.516684 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:47.517580 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:47.517805 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:47.517884 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:47.518134 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:47.518206 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:47.518260 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:47.518418 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 22:30:47.518468 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":127,"threads_started":1}
I20260811 22:30:47.525029 22850 raft_consensus.cc:1261] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Refusing update from remote peer 3d646afff3dd40a894fbdddf4da617a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:47.525054 23107 raft_consensus.cc:1261] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Refusing update from remote peer 3d646afff3dd40a894fbdddf4da617a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:47.525497 23280 consensus_queue.cc:1035] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f480fbc8cc734489b4790d07d613435c" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:47.525705 23376 consensus_queue.cc:1035] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f44cd316376c45ceb802c8828663b674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:30:47.539171 23343 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:30:49.093473 22334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:49.093652 22334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:30:49.096010 22334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99097b5118ad4c3e9f2fd6b4f74e0416: Sending DeleteTablet for 3 replicas of tablet 042e387083a44539be0b1d4a9fd01c3b
I20260811 22:30:49.096609 22568 tablet_service.cc:1514] Processing DeleteTablet for tablet 042e387083a44539be0b1d4a9fd01c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56528
I20260811 22:30:49.096647 22816 tablet_service.cc:1514] Processing DeleteTablet for tablet 042e387083a44539be0b1d4a9fd01c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55480
I20260811 22:30:49.096936 23068 tablet_service.cc:1514] Processing DeleteTablet for tablet 042e387083a44539be0b1d4a9fd01c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:30:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56186
I20260811 22:30:49.096971 23492 tablet_replica.cc:330] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0: stopping tablet replica
I20260811 22:30:49.097141 23492 raft_consensus.cc:2229] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:30:49.097281 23495 tablet_replica.cc:330] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: stopping tablet replica
I20260811 22:30:49.097400 23495 raft_consensus.cc:2229] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:49.097395 23492 raft_consensus.cc:2258] T 042e387083a44539be0b1d4a9fd01c3b P 3d646afff3dd40a894fbdddf4da617a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:49.097512 23495 raft_consensus.cc:2258] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:49.097643 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22457
I20260811 22:30:49.101140 23493 tablet_replica.cc:330] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: stopping tablet replica
I20260811 22:30:49.101306 23493 raft_consensus.cc:2229] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:30:49.101413 23493 raft_consensus.cc:2258] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:30:49.103610 23493 ts_tablet_manager.cc:1905] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:49.103814 23495 ts_tablet_manager.cc:1905] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:30:49.106634 23493 ts_tablet_manager.cc:1918] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:30:49.106716 23493 log.cc:1196] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/042e387083a44539be0b1d4a9fd01c3b
I20260811 22:30:49.106950 23493 ts_tablet_manager.cc:1939] T 042e387083a44539be0b1d4a9fd01c3b P f480fbc8cc734489b4790d07d613435c: Deleting consensus metadata
I20260811 22:30:49.107440 23495 ts_tablet_manager.cc:1918] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:30:49.107546 23495 log.cc:1196] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/042e387083a44539be0b1d4a9fd01c3b
I20260811 22:30:49.107875 23495 ts_tablet_manager.cc:1939] T 042e387083a44539be0b1d4a9fd01c3b P f44cd316376c45ceb802c8828663b674: Deleting consensus metadata
I20260811 22:30:49.108040 22315 catalog_manager.cc:4841] TS f480fbc8cc734489b4790d07d613435c (127.18.211.130:43791): tablet 042e387083a44539be0b1d4a9fd01c3b (table test-table [id=cf738a90e0af432193e746fff7ec4c41]) successfully deleted
I20260811 22:30:49.108741 22311 catalog_manager.cc:4841] TS f44cd316376c45ceb802c8828663b674 (127.18.211.131:33391): tablet 042e387083a44539be0b1d4a9fd01c3b (table test-table [id=cf738a90e0af432193e746fff7ec4c41]) successfully deleted
W20260811 22:30:49.122495 22315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:34929 (error 108)
I20260811 22:30:49.122826 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22698
I20260811 22:30:49.128166 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:49.125791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34929 (local address 127.0.0.1:56564)
0811 22:30:49.127898 (+ 2107us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:49.128017 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:34929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1997,"thread_start_us":1355,"threads_started":1}
W20260811 22:30:49.128396 22315 catalog_manager.cc:4584] TS 3d646afff3dd40a894fbdddf4da617a0 (127.18.211.129:34929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 042e387083a44539be0b1d4a9fd01c3b: Network error: Client connection negotiation failed: client connection to 127.18.211.129:34929: connect: Connection refused (error 111)
I20260811 22:30:49.135500 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22945
I20260811 22:30:49.149997 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22284
2026-08-11T22:30:49Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6797 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:49 dist-test-slave-8hhm krb5kdc[23508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:49 dist-test-slave-8hhm krb5kdc[23508](info): set up 2 sockets
Aug 11 22:30:49 dist-test-slave-8hhm krb5kdc[23508](info): commencing operation
krb5kdc: starting...
W20260811 22:30:50.464532 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:50 dist-test-slave-8hhm krb5kdc[23508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:50.595553 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:46019
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:46019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:50.714176 23536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:50.714478 23536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:50.714561 23536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:50.718547 23536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:50.718628 23536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:50.718664 23536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:50.718704 23536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:50.718742 23536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:50.724007 23536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:46019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:46019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:50.725091 23536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:50.726198 23536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:50.735342 23545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:50.735409 23549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:50.735343 23542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:50.735598 23536 server_base.cc:1034] running on GCE node
I20260811 22:30:50.736724 23536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:50.738373 23536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:50.739610 23536 hybrid_clock.cc:648] HybridClock initialized: now 1786487450739593 us; error 63 us; skew 500 ppm
Aug 11 22:30:50 dist-test-slave-8hhm krb5kdc[23508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:50.743553 23536 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:50.746323 23536 webserver.cc:460] Webserver started at http://127.18.211.190:35099/ using document root <none> and password file <none>
I20260811 22:30:50.747140 23536 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:50.747277 23536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:50.747568 23536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:50.749897 23536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "d4cd6ddeaa24443eabad0096aa99da5c"
format_stamp: "Formatted at 2026-08-11 22:30:50 on dist-test-slave-8hhm"
server_key: "0ab435501f1ef9c06f540d1a51156639"
server_key_iv: "fe2bf76f74aef573eaa547ba59e5c018"
server_key_version: "encryptionkey@0"
I20260811 22:30:50.750435 23536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "d4cd6ddeaa24443eabad0096aa99da5c"
format_stamp: "Formatted at 2026-08-11 22:30:50 on dist-test-slave-8hhm"
server_key: "0ab435501f1ef9c06f540d1a51156639"
server_key_iv: "fe2bf76f74aef573eaa547ba59e5c018"
server_key_version: "encryptionkey@0"
I20260811 22:30:50.754531 23536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:30:50.757143 23559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:50.758427 23536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:30:50.758574 23536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d4cd6ddeaa24443eabad0096aa99da5c"
format_stamp: "Formatted at 2026-08-11 22:30:50 on dist-test-slave-8hhm"
server_key: "0ab435501f1ef9c06f540d1a51156639"
server_key_iv: "fe2bf76f74aef573eaa547ba59e5c018"
server_key_version: "encryptionkey@0"
I20260811 22:30:50.758684 23536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:50.788209 23536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:50.791815 23536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:50.792063 23536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:50.799919 23536 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:46019
I20260811 22:30:50.799949 23641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:46019 every 8 connection(s)
I20260811 22:30:50.801008 23536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:50.803639 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23536
I20260811 22:30:50.803735 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:50.804040 19278 external_mini_cluster.cc:1428] Setting key 209e1f7a3534d3ea457e27307b3f4c13
I20260811 22:30:50.804141 23642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:50.812362 23642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c: Bootstrap starting.
Aug 11 22:30:50 dist-test-slave-8hhm krb5kdc[23508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:50.815096 23642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:50.815973 23642 log.cc:824] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:50.818495 23642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c: No bootstrap required, opened a new log
I20260811 22:30:50.820518 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:50.805618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60560 (local address 127.18.211.190:46019)
0811 22:30:50.806057 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:50.806076 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:50.806118 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:30:50.807222 (+ 1104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:50.807250 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:50.807722 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:50.808132 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:50.808980 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:50.810530 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:50.811449 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:50.812260 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:50.812458 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:50.815429 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:50.815434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:50.815449 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:50.815482 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:50.817480 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:50.817995 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:50.818000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:50.818005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:50.818120 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:50.818433 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:50.818435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:50.818436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:50.818916 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:50.819093 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:50.819412 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:30:50.819739 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":114,"threads_started":1}
I20260811 22:30:50.822893 23642 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46019 } }
I20260811 22:30:50.823166 23642 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:50.823262 23642 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4cd6ddeaa24443eabad0096aa99da5c, State: Initialized, Role: FOLLOWER
I20260811 22:30:50.823705 23642 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46019 } }
I20260811 22:30:50.823865 23642 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:50.823953 23642 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:50.824088 23642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:50.825062 23642 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46019 } }
I20260811 22:30:50.825461 23642 leader_election.cc:302] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4cd6ddeaa24443eabad0096aa99da5c; no voters:
I20260811 22:30:50.825835 23642 leader_election.cc:288] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:50.826051 23652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:50.826386 23652 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [term 1 LEADER]: Becoming Leader. State: Replica: d4cd6ddeaa24443eabad0096aa99da5c, State: Running, Role: LEADER
I20260811 22:30:50.826767 23652 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46019 } }
I20260811 22:30:50.827322 23642 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:50.828624 23655 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4cd6ddeaa24443eabad0096aa99da5c. Latest consensus state: current_term: 1 leader_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46019 } } }
I20260811 22:30:50.828776 23655 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:50.829191 23654 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cd6ddeaa24443eabad0096aa99da5c" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46019 } } }
I20260811 22:30:50.829332 23654 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:50.832988 23688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:50.833083 23688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:50.833190 23680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:50.834182 23680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:50.840426 23680 catalog_manager.cc:1347] Generated new cluster ID: 53a04863625f4d8c868b0852d4238c1f
I20260811 22:30:50.840515 23680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:50.854890 23680 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:50.856293 23680 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:50.863826 23680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4cd6ddeaa24443eabad0096aa99da5c: Generated new TSK 0
I20260811 22:30:50.864603 23680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:30:50.900504 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46019
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:51.021392 23702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:51.021677 23702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:51.021721 23702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:51.025867 23702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:51.025943 23702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:51.026003 23702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:30:51.031941 23702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:51.033144 23702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:51.034291 23702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:51.044816 23711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:51.044904 23702 server_base.cc:1034] running on GCE node
W20260811 22:30:51.044834 23710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:51.044833 23714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:51.045600 23702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:51.046288 23702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:51.047479 23702 hybrid_clock.cc:648] HybridClock initialized: now 1786487451047459 us; error 31 us; skew 500 ppm
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[23508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:51.050444 23702 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:30:51.051384 23702 webserver.cc:460] Webserver started at http://127.18.211.129:34685/ using document root <none> and password file <none>
I20260811 22:30:51.051999 23702 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:51.052094 23702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:51.052352 23702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:51.054566 23702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "de52baa6a5be46f4bfffd753450cbaf0"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "7d9efb94244b99bff8305b05a7bbac8e"
server_key_iv: "61bdef624a01baafae7540c45df21322"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.055169 23702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "de52baa6a5be46f4bfffd753450cbaf0"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "7d9efb94244b99bff8305b05a7bbac8e"
server_key_iv: "61bdef624a01baafae7540c45df21322"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.059177 23702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:51.061803 23725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.063140 23702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:30:51.063283 23702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "de52baa6a5be46f4bfffd753450cbaf0"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "7d9efb94244b99bff8305b05a7bbac8e"
server_key_iv: "61bdef624a01baafae7540c45df21322"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.063409 23702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:51.082468 23702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:51.085894 23702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:51.086158 23702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:51.086905 23702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:51.087818 23702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:51.087891 23702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.087966 23702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:51.088018 23702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.096926 23702 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:33285
I20260811 22:30:51.096951 23929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:33285 every 8 connection(s)
I20260811 22:30:51.097927 23702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:30:51.098848 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23702
I20260811 22:30:51.098940 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:30:51.099203 19278 external_mini_cluster.cc:1428] Setting key 57b4d1be0e61b395d21a712f8d9186a4
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[23508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:30:51.114315 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:51.100791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:44913 (local address 127.18.211.190:46019)
0811 22:30:51.100946 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:51.100951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:51.102060 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 22:30:51.103657 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:51.103664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:51.103729 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:51.103832 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:51.105060 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.106003 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.107198 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.107647 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.107782 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:51.110609 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:51.110613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:51.110618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:51.110651 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:51.112094 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:51.112580 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:51.112583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:51.112585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:51.112669 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:51.112991 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:51.113011 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:51.113014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:51.113334 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:51.113442 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:51.113964 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 22:30:51.114130 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:30:51.115264 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:51.101314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46019 (local address 127.18.211.129:44913)
0811 22:30:51.101871 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:51.101935 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:51.103430 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:51.103998 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:51.104005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:51.104403 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:51.104899 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:51.104913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.106154 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.106157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:51.107022 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:51.107031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.107801 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.107803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:51.107955 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:51.108616 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:51.108633 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:51.110432 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:51.112204 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:51.112209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:51.112222 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:51.112479 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:51.112753 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:51.112755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:51.112756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:51.112891 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:51.113454 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:51.113459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:51.113717 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 22:30:51.113920 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:30:51.114270 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":124,"threads_started":1}
I20260811 22:30:51.116628 23930 heartbeater.cc:347] Connected to a master server at 127.18.211.190:46019
I20260811 22:30:51.116892 23930 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:51.117467 23930 heartbeater.cc:511] Master 127.18.211.190:46019 requested a full tablet report, sending...
I20260811 22:30:51.119391 23589 ts_manager.cc:194] Registered new tserver with Master: de52baa6a5be46f4bfffd753450cbaf0 (127.18.211.129:33285)
I20260811 22:30:51.121156 23589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:44913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:30:51.131930 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46019
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:51.250952 23936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:51.251263 23936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:51.251310 23936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:51.255450 23936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:51.255524 23936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:51.255586 23936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:30:51.260943 23936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:51.262168 23936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:51.263370 23936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:51.273845 23944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:51.273845 23948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:51.273851 23945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:51.274094 23936 server_base.cc:1034] running on GCE node
I20260811 22:30:51.274677 23936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:51.275451 23936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:51.276652 23936 hybrid_clock.cc:648] HybridClock initialized: now 1786487451276634 us; error 33 us; skew 500 ppm
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[23508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:51.279645 23936 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:30:51.280596 23936 webserver.cc:460] Webserver started at http://127.18.211.130:42905/ using document root <none> and password file <none>
I20260811 22:30:51.281198 23936 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:51.281352 23936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:51.281608 23936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:51.283756 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "ad641cb3c2fa413f9ee2f60d81b0e23a"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "67465811c0a7e83f734c12a0ce588b83"
server_key_iv: "37d097a54a3adaac1bdf62ccefec5b7c"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.284374 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "ad641cb3c2fa413f9ee2f60d81b0e23a"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "67465811c0a7e83f734c12a0ce588b83"
server_key_iv: "37d097a54a3adaac1bdf62ccefec5b7c"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.288285 23936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 22:30:51.290804 23959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.292061 23936 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:30:51.292204 23936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "ad641cb3c2fa413f9ee2f60d81b0e23a"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "67465811c0a7e83f734c12a0ce588b83"
server_key_iv: "37d097a54a3adaac1bdf62ccefec5b7c"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.292320 23936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:51.307827 23936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:51.311403 23936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:51.311635 23936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:51.312305 23936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:51.313187 23936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:51.313311 23936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.313387 23936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:51.313438 23936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.322510 23936 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:42529
I20260811 22:30:51.322540 24182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:42529 every 8 connection(s)
I20260811 22:30:51.323453 23936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:30:51.330117 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23936
I20260811 22:30:51.330205 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:30:51.330483 19278 external_mini_cluster.cc:1428] Setting key 4d6c723bea8dc2155966388ae472a1a9
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[23508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:51.338429 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:51.325423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:49089 (local address 127.18.211.190:46019)
0811 22:30:51.325582 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:51.325586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:51.326691 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 22:30:51.327675 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:51.327678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:51.327728 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:51.327805 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:51.329158 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.329778 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.331176 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.331431 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.331519 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:51.334633 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:51.334636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:51.334638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:51.334665 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:51.336123 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:51.336688 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:51.336696 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:51.336697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:51.336777 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:51.337184 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:51.337191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:51.337193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:51.337432 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:51.337559 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:51.338111 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 22:30:51.338259 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:30:51.339305 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:51.325872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46019 (local address 127.18.211.130:49089)
0811 22:30:51.326538 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:51.326584 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:51.327480 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:51.328027 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:51.328037 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:51.328465 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:51.328980 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:51.328993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.329943 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.329946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:51.331057 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:51.331074 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.331613 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.331618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:51.331856 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:51.332589 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:51.332607 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:51.334444 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:51.336254 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:51.336260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:51.336279 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:51.336561 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:51.336884 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:51.336887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:51.336889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:51.337088 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:51.337549 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:51.337556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:51.337841 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 22:30:51.338067 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 22:30:51.338438 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":120,"threads_started":1}
I20260811 22:30:51.340690 24183 heartbeater.cc:347] Connected to a master server at 127.18.211.190:46019
I20260811 22:30:51.340955 24183 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:30:51.341562 24183 heartbeater.cc:511] Master 127.18.211.190:46019 requested a full tablet report, sending...
I20260811 22:30:51.342805 23589 ts_manager.cc:194] Registered new tserver with Master: ad641cb3c2fa413f9ee2f60d81b0e23a (127.18.211.130:42529)
I20260811 22:30:51.343382 23589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:49089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:30:51.362505 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46019
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:51.484699 24192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:51.485050 24192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:51.485150 24192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:51.489494 24192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:51.489588 24192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:51.489683 24192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:30:51.495115 24192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:51.496335 24192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:51.497589 24192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:51.507884 24201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:51.507879 24206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:51.507937 24203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:51.508203 24192 server_base.cc:1034] running on GCE node
I20260811 22:30:51.508641 24192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:51.509413 24192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:51.510579 24192 hybrid_clock.cc:648] HybridClock initialized: now 1786487451510546 us; error 45 us; skew 500 ppm
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[23508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:51.513773 24192 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:30:51.514789 24192 webserver.cc:460] Webserver started at http://127.18.211.131:38011/ using document root <none> and password file <none>
I20260811 22:30:51.515381 24192 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:51.515446 24192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:51.515707 24192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:51.517901 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "deff609485994636b8eae71b45b497e8"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "04f32f479f95404e012838997d145bec"
server_key_iv: "a1a64dfff3a58a70943240ae8c4d07fc"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.518488 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "deff609485994636b8eae71b45b497e8"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "04f32f479f95404e012838997d145bec"
server_key_iv: "a1a64dfff3a58a70943240ae8c4d07fc"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.522498 24192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:30:51.525166 24224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.526546 24192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:51.526717 24192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "deff609485994636b8eae71b45b497e8"
format_stamp: "Formatted at 2026-08-11 22:30:51 on dist-test-slave-8hhm"
server_key: "04f32f479f95404e012838997d145bec"
server_key_iv: "a1a64dfff3a58a70943240ae8c4d07fc"
server_key_version: "encryptionkey@0"
I20260811 22:30:51.526846 24192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:51.553709 24192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:51.557483 24192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:51.557720 24192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:51.558434 24192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:51.559353 24192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:51.559427 24192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.559502 24192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:51.559559 24192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:51.569686 24192 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:41191
I20260811 22:30:51.569746 24444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:41191 every 8 connection(s)
I20260811 22:30:51.570802 24192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[23508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:51.580943 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24192
I20260811 22:30:51.581046 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:30:51.581437 19278 external_mini_cluster.cc:1428] Setting key 2ed9056db5bf6a642b0212b3573e71c6
I20260811 22:30:51.585502 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:51.572808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:56145 (local address 127.18.211.190:46019)
0811 22:30:51.572962 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:51.572966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:51.574149 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 22:30:51.575034 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:51.575036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:51.575092 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:51.575172 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:51.576476 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.577067 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.578293 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.578532 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.578628 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:51.581240 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:51.581245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:51.581251 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:51.581285 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:51.583115 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:51.583768 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:51.583771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:51.583776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:51.583875 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:51.584284 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:51.584289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:51.584292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:51.584543 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:51.584640 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:51.585135 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 22:30:51.585313 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:30:51.586376 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:51.573256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46019 (local address 127.18.211.131:56145)
0811 22:30:51.573971 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:51.574018 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:51.574837 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:51.575360 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:51.575367 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:51.575770 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:51.576287 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:51.576302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.577197 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.577200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:51.578171 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:51.578183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:51.578696 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:51.578698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:51.578852 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:51.579429 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:51.579447 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:51.580974 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:51.583233 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:51.583240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:51.583258 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:51.583649 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:51.583980 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:51.583983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:51.583985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:51.584182 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:51.584647 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:51.584652 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:51.584907 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 22:30:51.585077 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 22:30:51.585432 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":124,"threads_started":1}
I20260811 22:30:51.587710 24448 heartbeater.cc:347] Connected to a master server at 127.18.211.190:46019
I20260811 22:30:51.588012 24448 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:51.588599 24448 heartbeater.cc:511] Master 127.18.211.190:46019 requested a full tablet report, sending...
I20260811 22:30:51.589749 23589 ts_manager.cc:194] Registered new tserver with Master: deff609485994636b8eae71b45b497e8 (127.18.211.131:41191)
I20260811 22:30:51.590334 23589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:56145
I20260811 22:30:51.596166 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:30:51.613727 23650 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:51.612019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.18.211.190:46019)
0811 22:30:51.612199 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:51.612204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:51.612355 (+ 151us) server_negotiation.cc:407] Connection header received
0811 22:30:51.612476 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:51.612479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:51.612526 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:51.612602 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:51.613574 (+ 972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48522: BlockingRecv error: recv got EOF from 127.0.0.1:48522 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:30:51.614614 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23702
I20260811 22:30:51.634912 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23936
I20260811 22:30:51.654279 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24192
I20260811 22:30:51.669754 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23536
2026-08-11T22:30:51Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2515 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[24465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[24465](info): set up 2 sockets
Aug 11 22:30:51 dist-test-slave-8hhm krb5kdc[24465](info): commencing operation
krb5kdc: starting...
W20260811 22:30:52.977396 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:53 dist-test-slave-8hhm krb5kdc[24465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:53.109992 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:37975
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:53.227108 24486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:53.227378 24486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:53.227419 24486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:53.231413 24486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:53.231467 24486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:53.231487 24486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:53.231503 24486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:53.231518 24486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:53.236776 24486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:37975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:53.237877 24486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:53.238881 24486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:53.247685 24503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:53.247715 24495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:53.247684 24496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:53.247942 24486 server_base.cc:1034] running on GCE node
I20260811 22:30:53.248622 24486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:53.249904 24486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:53.251097 24486 hybrid_clock.cc:648] HybridClock initialized: now 1786487453251078 us; error 33 us; skew 500 ppm
Aug 11 22:30:53 dist-test-slave-8hhm krb5kdc[24465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487453, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:53.254052 24486 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:53.255084 24486 webserver.cc:460] Webserver started at http://127.18.211.190:36613/ using document root <none> and password file <none>
I20260811 22:30:53.255726 24486 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:53.255832 24486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:53.256108 24486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:53.258515 24486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "1fc605dde1874735b5d1cfa92feff6d6"
format_stamp: "Formatted at 2026-08-11 22:30:53 on dist-test-slave-8hhm"
server_key: "df2a6167d35ede17034296f8578e9eca"
server_key_iv: "a681da0a3c129984ebe623390a679907"
server_key_version: "encryptionkey@0"
I20260811 22:30:53.259188 24486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "1fc605dde1874735b5d1cfa92feff6d6"
format_stamp: "Formatted at 2026-08-11 22:30:53 on dist-test-slave-8hhm"
server_key: "df2a6167d35ede17034296f8578e9eca"
server_key_iv: "a681da0a3c129984ebe623390a679907"
server_key_version: "encryptionkey@0"
I20260811 22:30:53.263729 24486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:53.266414 24515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:53.267778 24486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:30:53.267935 24486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "1fc605dde1874735b5d1cfa92feff6d6"
format_stamp: "Formatted at 2026-08-11 22:30:53 on dist-test-slave-8hhm"
server_key: "df2a6167d35ede17034296f8578e9eca"
server_key_iv: "a681da0a3c129984ebe623390a679907"
server_key_version: "encryptionkey@0"
I20260811 22:30:53.268064 24486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:53.282145 24486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:53.285359 24486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:53.285573 24486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:53.293699 24486 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:37975
I20260811 22:30:53.294013 24629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:37975 every 8 connection(s)
I20260811 22:30:53.294776 24486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:53.297734 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24486
I20260811 22:30:53.297837 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:53.298082 24631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:53.298130 19278 external_mini_cluster.cc:1428] Setting key f5004b4df974f43d2968bcd27da4b4e0
I20260811 22:30:53.306427 24631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6: Bootstrap starting.
Aug 11 22:30:53 dist-test-slave-8hhm krb5kdc[24465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:53.309180 24631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:53.310241 24631 log.cc:824] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:53.312718 24631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6: No bootstrap required, opened a new log
I20260811 22:30:53.314906 24635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:53.299488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.18.211.190:37975)
0811 22:30:53.299921 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:53.299937 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:53.299974 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:30:53.300956 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:53.300976 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:53.301391 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:53.301692 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:53.302854 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:53.304478 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:53.305963 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:53.306455 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:53.306643 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:53.309572 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:53.309579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:53.309596 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:53.309631 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:53.311662 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:53.312126 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:53.312133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:53.312139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:53.312287 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:53.312574 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:53.312578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:53.312580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:53.313123 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:53.313387 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:53.313605 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:30:53.313921 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":144,"threads_started":1}
I20260811 22:30:53.317509 24631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc605dde1874735b5d1cfa92feff6d6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37975 } }
I20260811 22:30:53.317806 24631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:53.317890 24631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc605dde1874735b5d1cfa92feff6d6, State: Initialized, Role: FOLLOWER
I20260811 22:30:53.318197 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:53.318434 24631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc605dde1874735b5d1cfa92feff6d6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37975 } }
I20260811 22:30:53.318599 24631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:53.318670 24631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:53.318800 24631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:53.319792 24631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc605dde1874735b5d1cfa92feff6d6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37975 } }
I20260811 22:30:53.320181 24631 leader_election.cc:302] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fc605dde1874735b5d1cfa92feff6d6; no voters:
I20260811 22:30:53.320554 24631 leader_election.cc:288] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:53.320681 24641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:53.320991 24641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [term 1 LEADER]: Becoming Leader. State: Replica: 1fc605dde1874735b5d1cfa92feff6d6, State: Running, Role: LEADER
I20260811 22:30:53.321409 24641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc605dde1874735b5d1cfa92feff6d6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37975 } }
I20260811 22:30:53.322232 24631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:53.323092 24650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fc605dde1874735b5d1cfa92feff6d6. Latest consensus state: current_term: 1 leader_uuid: "1fc605dde1874735b5d1cfa92feff6d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc605dde1874735b5d1cfa92feff6d6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37975 } } }
I20260811 22:30:53.323238 24650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:53.323568 24646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fc605dde1874735b5d1cfa92feff6d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc605dde1874735b5d1cfa92feff6d6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37975 } } }
I20260811 22:30:53.323657 24646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6 [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:53.327931 24681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:53.328029 24681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:53.328145 24659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:53.328852 24659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:53.334419 24659 catalog_manager.cc:1347] Generated new cluster ID: c60cb176547e4a37b9110acbd9c2b795
I20260811 22:30:53.334556 24659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:53.342005 24659 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:53.342886 24659 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:53.349205 24635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:53.341978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.18.211.190:37975)
0811 22:30:53.342084 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:53.342088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:53.343684 (+ 1596us) server_negotiation.cc:407] Connection header received
0811 22:30:53.343889 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:53.343892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:53.343941 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:53.344028 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:53.344625 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:53.345461 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:53.346701 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:53.347036 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:53.347122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:53.348102 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:53.348106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:53.348109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:53.348703 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:53.348741 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:53.348763 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:53.348857 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:53.348979 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 22:30:53.349047 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 22:30:53.350075 24659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fc605dde1874735b5d1cfa92feff6d6: Generated new TSK 0
I20260811 22:30:53.350849 24659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:53.357545 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24486
2026-08-11T22:30:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1690 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:53 dist-test-slave-8hhm krb5kdc[24703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:53 dist-test-slave-8hhm krb5kdc[24703](info): set up 2 sockets
Aug 11 22:30:53 dist-test-slave-8hhm krb5kdc[24703](info): commencing operation
krb5kdc: starting...
W20260811 22:30:54.697088 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:54 dist-test-slave-8hhm krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:54.832944 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44577
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:44577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:54.954299 24735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:54.954579 24735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:54.954620 24735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:54.958870 24735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:54.958940 24735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:54.958962 24735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:54.959007 24735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:54.959039 24735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:54.964485 24735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:44577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:54.965641 24735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:54.966724 24735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:54.975754 24753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:54.975811 24735 server_base.cc:1034] running on GCE node
W20260811 22:30:54.975819 24747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:54.975997 24744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:54.976626 24735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:54.977833 24735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:54.979031 24735 hybrid_clock.cc:648] HybridClock initialized: now 1786487454979009 us; error 30 us; skew 500 ppm
Aug 11 22:30:54 dist-test-slave-8hhm krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487454, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:54.987185 24735 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:54.988149 24735 webserver.cc:460] Webserver started at http://127.18.211.190:36333/ using document root <none> and password file <none>
I20260811 22:30:54.988714 24735 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:54.988777 24735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:54.988955 24735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:54.991200 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "649e1ae277d94024b1793981382434c8"
format_stamp: "Formatted at 2026-08-11 22:30:54 on dist-test-slave-8hhm"
server_key: "661bc40b040fa840683cc1e7ebcec0d9"
server_key_iv: "b6c38be1f4aa4579d0d5a7e07a3e4a29"
server_key_version: "encryptionkey@0"
I20260811 22:30:54.991825 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "649e1ae277d94024b1793981382434c8"
format_stamp: "Formatted at 2026-08-11 22:30:54 on dist-test-slave-8hhm"
server_key: "661bc40b040fa840683cc1e7ebcec0d9"
server_key_iv: "b6c38be1f4aa4579d0d5a7e07a3e4a29"
server_key_version: "encryptionkey@0"
I20260811 22:30:54.995808 24735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:54.998407 24766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:54.999718 24735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:54.999825 24735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "649e1ae277d94024b1793981382434c8"
format_stamp: "Formatted at 2026-08-11 22:30:54 on dist-test-slave-8hhm"
server_key: "661bc40b040fa840683cc1e7ebcec0d9"
server_key_iv: "b6c38be1f4aa4579d0d5a7e07a3e4a29"
server_key_version: "encryptionkey@0"
I20260811 22:30:54.999917 24735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:55.027077 24735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:55.029978 24735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:55.030136 24735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:55.037721 24735 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:44577
I20260811 22:30:55.038066 24883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:44577 every 8 connection(s)
I20260811 22:30:55.038802 24735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:55.040921 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24735
I20260811 22:30:55.041024 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:55.041375 19278 external_mini_cluster.cc:1428] Setting key 4c31ee212e25826a4216ebcdc1e4eaf3
I20260811 22:30:55.042060 24885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:55.048125 24885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8: Bootstrap starting.
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:55.050662 24885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:55.051661 24885 log.cc:824] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:55.054126 24885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8: No bootstrap required, opened a new log
I20260811 22:30:55.056270 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.042871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.18.211.190:44577)
0811 22:30:55.043322 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:55.043338 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:55.043375 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:30:55.044054 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:55.044071 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:55.044386 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:55.044732 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:55.045603 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.046603 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.047467 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.047961 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.048143 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.050939 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:55.050947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:55.050968 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:55.051003 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:55.052987 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.053513 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.053520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.053525 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.053665 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.053954 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.053957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.053959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.054481 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:55.054692 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:55.054970 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 22:30:55.055277 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":145,"threads_started":1}
I20260811 22:30:55.058631 24885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e1ae277d94024b1793981382434c8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44577 } }
I20260811 22:30:55.058915 24885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:55.058959 24885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649e1ae277d94024b1793981382434c8, State: Initialized, Role: FOLLOWER
I20260811 22:30:55.059394 24885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e1ae277d94024b1793981382434c8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44577 } }
I20260811 22:30:55.059521 24885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:55.059569 24885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:55.059643 24885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:55.060524 24885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e1ae277d94024b1793981382434c8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44577 } }
I20260811 22:30:55.060827 24885 leader_election.cc:302] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 649e1ae277d94024b1793981382434c8; no voters:
I20260811 22:30:55.061151 24885 leader_election.cc:288] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:55.061308 24895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:55.061583 24895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [term 1 LEADER]: Becoming Leader. State: Replica: 649e1ae277d94024b1793981382434c8, State: Running, Role: LEADER
I20260811 22:30:55.061983 24895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e1ae277d94024b1793981382434c8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44577 } }
I20260811 22:30:55.062517 24885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:55.063715 24897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "649e1ae277d94024b1793981382434c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e1ae277d94024b1793981382434c8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44577 } } }
I20260811 22:30:55.063858 24897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:55.064247 24913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:55.064208 24899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 649e1ae277d94024b1793981382434c8. Latest consensus state: current_term: 1 leader_uuid: "649e1ae277d94024b1793981382434c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e1ae277d94024b1793981382434c8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44577 } } }
I20260811 22:30:55.064383 24899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:55.067543 24913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:55.073185 24913 catalog_manager.cc:1347] Generated new cluster ID: 63ccbce4d06342848c76ec7e5770dbd9
I20260811 22:30:55.073300 24913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:55.090122 24913 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:55.091487 24913 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:55.097771 24913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 649e1ae277d94024b1793981382434c8: Generated new TSK 0
I20260811 22:30:55.098486 24913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:30:55.132436 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44577
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:55.250162 24935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:55.250420 24935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:55.250463 24935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:55.254452 24935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:55.254528 24935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:55.254633 24935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:30:55.260039 24935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:55.261183 24935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:55.262285 24935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:55.272368 24944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:55.272537 24949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:55.272574 24935 server_base.cc:1034] running on GCE node
W20260811 22:30:55.272368 24947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:55.273103 24935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:55.273761 24935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:55.275022 24935 hybrid_clock.cc:648] HybridClock initialized: now 1786487455275007 us; error 29 us; skew 500 ppm
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:55.277977 24935 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:30:55.278901 24935 webserver.cc:460] Webserver started at http://127.18.211.129:37623/ using document root <none> and password file <none>
I20260811 22:30:55.279475 24935 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:55.279570 24935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:55.279819 24935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:55.281935 24935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "140ea3437e06432a88df8fe58d51d7b3"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "bd0525d40526e24fee11b1c972e950a9"
server_key_iv: "e53e94cc0fbb794b5c73efacdd94008a"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.282506 24935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "140ea3437e06432a88df8fe58d51d7b3"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "bd0525d40526e24fee11b1c972e950a9"
server_key_iv: "e53e94cc0fbb794b5c73efacdd94008a"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.286274 24935 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:55.288779 24970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.290033 24935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:30:55.290194 24935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "140ea3437e06432a88df8fe58d51d7b3"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "bd0525d40526e24fee11b1c972e950a9"
server_key_iv: "e53e94cc0fbb794b5c73efacdd94008a"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.290314 24935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:55.306226 24935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:55.309855 24935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:55.310065 24935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:55.310698 24935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:55.311564 24935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:55.311638 24935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.311736 24935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:55.311789 24935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.323144 24935 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:35823
I20260811 22:30:55.323164 25213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:35823 every 8 connection(s)
I20260811 22:30:55.324136 24935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:30:55.330214 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24935
I20260811 22:30:55.330317 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:30:55.330622 19278 external_mini_cluster.cc:1428] Setting key 972f0ffe2f0cc865c43b9be358c37a83
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:55.340111 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.326018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:39821 (local address 127.18.211.190:44577)
0811 22:30:55.326166 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:55.326169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:55.326987 (+ 818us) server_negotiation.cc:407] Connection header received
0811 22:30:55.328502 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:55.328505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:55.328567 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:55.328678 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:55.329969 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.330626 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.332231 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.332458 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.332542 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.335958 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:55.335961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:55.335963 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:55.335989 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:55.337650 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.338333 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.338336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.338337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.338413 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.338860 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.338862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.338864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.339051 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:55.339129 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:55.339866 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 22:30:55.339980 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:30:55.341490 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.326394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44577 (local address 127.18.211.129:39821)
0811 22:30:55.326838 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:55.326881 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:55.328278 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:55.328866 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:55.328873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:55.329311 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:55.329812 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:55.329826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.330815 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.330819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:55.332089 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:55.332103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.332629 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.332637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:55.332889 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.333607 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:55.333634 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:55.335766 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:55.337791 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:55.337800 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:55.337822 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:55.338220 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:55.338513 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:55.338519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:55.338523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:55.338750 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:55.339167 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:55.339178 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:55.339577 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 22:30:55.339833 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 22:30:55.340242 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:30:55.342911 25215 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44577
I20260811 22:30:55.343140 25215 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:55.343734 25215 heartbeater.cc:511] Master 127.18.211.190:44577 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:30:55.345450 24800 ts_manager.cc:194] Registered new tserver with Master: 140ea3437e06432a88df8fe58d51d7b3 (127.18.211.129:35823)
I20260811 22:30:55.346690 24800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:39821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:30:55.364912 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44577
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:55.481582 25220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:55.481948 25220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:55.482038 25220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:55.486091 25220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:55.486202 25220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:55.486301 25220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:30:55.491822 25220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:55.493057 25220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:55.494232 25220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:30:55.504629 25220 server_base.cc:1034] running on GCE node
W20260811 22:30:55.504598 25235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:55.504648 25228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:55.504885 25227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:55.505391 25220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:55.506086 25220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:55.507251 25220 hybrid_clock.cc:648] HybridClock initialized: now 1786487455507228 us; error 30 us; skew 500 ppm
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:55.510151 25220 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:30:55.511061 25220 webserver.cc:460] Webserver started at http://127.18.211.130:46629/ using document root <none> and password file <none>
I20260811 22:30:55.511612 25220 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:55.511672 25220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:55.511879 25220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:55.513989 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "bf1d1961071f47c790adc0c5b532d54c"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "45b05e6d40c817fd688904f32cd5ca34"
server_key_iv: "c457ade44cf6124702078a2722dc937a"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.514523 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "bf1d1961071f47c790adc0c5b532d54c"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "45b05e6d40c817fd688904f32cd5ca34"
server_key_iv: "c457ade44cf6124702078a2722dc937a"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.518325 25220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:55.521590 25247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.522993 25220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:30:55.523139 25220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "bf1d1961071f47c790adc0c5b532d54c"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "45b05e6d40c817fd688904f32cd5ca34"
server_key_iv: "c457ade44cf6124702078a2722dc937a"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.523270 25220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:55.543481 25220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:55.546789 25220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:55.547030 25220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:55.547683 25220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:55.548573 25220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:55.548645 25220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.548719 25220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:55.548770 25220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.558108 25220 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:46433
I20260811 22:30:55.558136 25474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:46433 every 8 connection(s)
I20260811 22:30:55.559103 25220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:30:55.563311 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25220
I20260811 22:30:55.563436 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:30:55.563858 19278 external_mini_cluster.cc:1428] Setting key 6f9a74476ae23dd742a32ed906ffe01e
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:30:55.577567 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.561100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:44167 (local address 127.18.211.190:44577)
0811 22:30:55.561304 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:55.561308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:55.563473 (+ 2165us) server_negotiation.cc:407] Connection header received
0811 22:30:55.565378 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:55.565381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:55.565446 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:55.565514 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:55.566798 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.567662 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.569595 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.569822 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.569912 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.573325 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:55.573331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:55.573338 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:55.573379 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:55.575264 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.575817 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.575821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.575822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.575925 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.576299 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.576305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.576309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.576579 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:55.576693 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:55.577324 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 22:30:55.577437 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:30:55.578603 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.561650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44577 (local address 127.18.211.130:44167)
0811 22:30:55.563302 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:55.563360 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:55.565088 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:55.565708 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:55.565716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:55.566110 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:55.566628 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:55.566643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.567827 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.567832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:55.569451 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:55.569467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.570015 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.570023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:55.570355 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.571079 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:55.571110 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:55.573082 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:55.575389 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:55.575395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:55.575411 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:55.575703 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:55.576032 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:55.576034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:55.576036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:55.576187 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:55.576701 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:55.576706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:55.577037 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 22:30:55.577269 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 22:30:55.577574 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1423,"thread_start_us":1224,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:30:55.580060 25475 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44577
I20260811 22:30:55.580377 25475 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:55.581019 25475 heartbeater.cc:511] Master 127.18.211.190:44577 requested a full tablet report, sending...
I20260811 22:30:55.582369 24800 ts_manager.cc:194] Registered new tserver with Master: bf1d1961071f47c790adc0c5b532d54c (127.18.211.130:46433)
I20260811 22:30:55.582994 24800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:44167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:30:55.596531 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44577
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:55.715054 25490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:55.715337 25490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:55.715381 25490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:55.719352 25490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:55.719410 25490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:55.719470 25490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:30:55.724728 25490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:55.725766 25490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:55.726810 25490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:55.737161 25497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:55.737157 25498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:55.737197 25506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:55.737720 25490 server_base.cc:1034] running on GCE node
I20260811 22:30:55.738149 25490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:55.738845 25490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:55.740041 25490 hybrid_clock.cc:648] HybridClock initialized: now 1786487455740026 us; error 31 us; skew 500 ppm
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:55.742857 25490 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:30:55.743824 25490 webserver.cc:460] Webserver started at http://127.18.211.131:35949/ using document root <none> and password file <none>
I20260811 22:30:55.744403 25490 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:55.744500 25490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:55.744750 25490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:55.746941 25490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "eb167f0532fa4bad849d9a0dbe8c4f26"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "7ef4b304391fec0bd0f79e6ba9e29072"
server_key_iv: "086390fba57311f1f76ec4e37d01513b"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.747521 25490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "eb167f0532fa4bad849d9a0dbe8c4f26"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "7ef4b304391fec0bd0f79e6ba9e29072"
server_key_iv: "086390fba57311f1f76ec4e37d01513b"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.753260 25490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 22:30:55.758203 25516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.759693 25490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 22:30:55.759851 25490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "eb167f0532fa4bad849d9a0dbe8c4f26"
format_stamp: "Formatted at 2026-08-11 22:30:55 on dist-test-slave-8hhm"
server_key: "7ef4b304391fec0bd0f79e6ba9e29072"
server_key_iv: "086390fba57311f1f76ec4e37d01513b"
server_key_version: "encryptionkey@0"
I20260811 22:30:55.759994 25490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:55.798355 25490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:55.801522 25490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:55.801744 25490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:55.802390 25490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:55.803305 25490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:55.803383 25490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.803490 25490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:55.803531 25490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:55.812520 25490 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:41565
I20260811 22:30:55.812549 25738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:41565 every 8 connection(s)
I20260811 22:30:55.813473 25490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:30:55.814849 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25490
I20260811 22:30:55.814975 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:30:55.815294 19278 external_mini_cluster.cc:1428] Setting key 54de992e1335c621faddb44183c8ba58
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:55.827657 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.815441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:56303 (local address 127.18.211.190:44577)
0811 22:30:55.815620 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:55.815627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:55.816656 (+ 1029us) server_negotiation.cc:407] Connection header received
0811 22:30:55.818362 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:55.818365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:55.818421 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:55.818499 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:55.819694 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.820237 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.821472 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.821700 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.821814 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.824251 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:55.824254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:55.824256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:55.824280 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:55.825765 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.826214 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.826217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.826218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.826291 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.826583 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.826585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.826586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.826841 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:55.826923 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:55.827435 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 22:30:55.827533 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:30:55.828843 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.815978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44577 (local address 127.18.211.131:56303)
0811 22:30:55.816503 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:55.816565 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:55.818152 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:55.818689 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:55.818696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:55.819065 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:55.819552 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:55.819565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.820367 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.820371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:55.821346 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:55.821356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.821853 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.821858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:55.822035 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.822610 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:55.822626 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:55.824086 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:55.825867 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:55.825871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:55.825883 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:55.826127 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:55.826371 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:55.826373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:55.826375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:55.826506 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:55.826937 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:55.826942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:55.827221 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 22:30:55.827385 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:30:55.827634 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":108,"threads_started":1}
I20260811 22:30:55.829900 25740 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44577
I20260811 22:30:55.830125 25740 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:55.830585 25740 heartbeater.cc:511] Master 127.18.211.190:44577 requested a full tablet report, sending...
I20260811 22:30:55.831534 24800 ts_manager.cc:194] Registered new tserver with Master: eb167f0532fa4bad849d9a0dbe8c4f26 (127.18.211.131:41565)
I20260811 22:30:55.832137 24800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:56303
I20260811 22:30:55.841604 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[24703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487455, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:55.870198 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:55.860968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.18.211.190:44577)
0811 22:30:55.861143 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:55.861148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:55.861483 (+ 335us) server_negotiation.cc:407] Connection header received
0811 22:30:55.861685 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:55.861687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:55.861734 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:55.861822 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:55.862608 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.863151 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.864110 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:55.864336 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:55.864446 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:55.867085 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:55.867089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:55.867091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:55.867117 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:55.868359 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.868796 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.868804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.868806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.868885 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:55.869254 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:55.869263 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:55.869265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:55.869496 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:55.869651 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:55.869896 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:30:55.870021 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 22:30:55.870628 24800 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56238
I20260811 22:30:55.872007 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24935
I20260811 22:30:55.887935 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25220
I20260811 22:30:55.903071 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25490
I20260811 22:30:55.909077 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24735
2026-08-11T22:30:55Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2547 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[25765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[25765](info): set up 2 sockets
Aug 11 22:30:55 dist-test-slave-8hhm krb5kdc[25765](info): commencing operation
krb5kdc: starting...
W20260811 22:30:57.222019 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:30:57 dist-test-slave-8hhm krb5kdc[25765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:30:57.356958 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42271
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:57.488291 25794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:57.488626 25794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:57.488708 25794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:57.493013 25794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:57.493093 25794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:57.493112 25794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:57.493175 25794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:57.493228 25794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:57.498983 25794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:42271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:57.500231 25794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:57.501454 25794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:57.510251 25809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:57.510236 25804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:57.510236 25803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:57.510614 25794 server_base.cc:1034] running on GCE node
I20260811 22:30:57.511308 25794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:57.512465 25794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:57.513655 25794 hybrid_clock.cc:648] HybridClock initialized: now 1786487457513643 us; error 30 us; skew 500 ppm
Aug 11 22:30:57 dist-test-slave-8hhm krb5kdc[25765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487457, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:57.516934 25794 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:30:57.517887 25794 webserver.cc:460] Webserver started at http://127.18.211.190:43707/ using document root <none> and password file <none>
I20260811 22:30:57.518426 25794 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:57.518487 25794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:57.518659 25794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:57.520763 25794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "3af9ca9a51df453dbb11679827f5e962"
format_stamp: "Formatted at 2026-08-11 22:30:57 on dist-test-slave-8hhm"
server_key: "1e32dedeea8c944c83c60ea7e0b50543"
server_key_iv: "35348814e1aeade78e38a35758693939"
server_key_version: "encryptionkey@0"
I20260811 22:30:57.521399 25794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "3af9ca9a51df453dbb11679827f5e962"
format_stamp: "Formatted at 2026-08-11 22:30:57 on dist-test-slave-8hhm"
server_key: "1e32dedeea8c944c83c60ea7e0b50543"
server_key_iv: "35348814e1aeade78e38a35758693939"
server_key_version: "encryptionkey@0"
I20260811 22:30:57.525293 25794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:57.527796 25823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:57.529065 25794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:30:57.529212 25794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "3af9ca9a51df453dbb11679827f5e962"
format_stamp: "Formatted at 2026-08-11 22:30:57 on dist-test-slave-8hhm"
server_key: "1e32dedeea8c944c83c60ea7e0b50543"
server_key_iv: "35348814e1aeade78e38a35758693939"
server_key_version: "encryptionkey@0"
I20260811 22:30:57.529350 25794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:57.545468 25794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:57.548502 25794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:57.548744 25794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:57.556703 25794 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:42271
I20260811 22:30:57.556721 25926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:42271 every 8 connection(s)
I20260811 22:30:57.557824 25794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:57.561096 25928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:57.565537 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25794
I20260811 22:30:57.565702 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:30:57.566052 19278 external_mini_cluster.cc:1428] Setting key 3418f4f4c0a6be66a9ec248dca9f2f69
I20260811 22:30:57.567723 25928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962: Bootstrap starting.
I20260811 22:30:57.570765 25928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:57.571813 25928 log.cc:824] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:57.574151 25928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962: No bootstrap required, opened a new log
Aug 11 22:30:57 dist-test-slave-8hhm krb5kdc[25765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:57.578908 25928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9ca9a51df453dbb11679827f5e962" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42271 } }
I20260811 22:30:57.579243 25928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:57.579320 25928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3af9ca9a51df453dbb11679827f5e962, State: Initialized, Role: FOLLOWER
I20260811 22:30:57.579854 25928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9ca9a51df453dbb11679827f5e962" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42271 } }
I20260811 22:30:57.580050 25928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:57.580132 25928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:57.580255 25928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:57.583225 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:57.567703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.18.211.190:42271)
0811 22:30:57.568221 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:57.568240 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:57.568285 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:30:57.569370 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:57.569397 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:57.569864 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:57.570257 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:57.571020 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:57.572498 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:57.573489 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:57.574208 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:57.574422 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:57.577016 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:57.577025 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:57.577050 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:57.577094 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:57.579236 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:57.579794 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:57.579800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:57.579806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:57.579977 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:57.580403 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:57.580406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:57.580407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:57.580952 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:57.581167 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:57.581478 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:30:57.582419 (+ 941us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":155,"threads_started":1}
I20260811 22:30:57.592391 25928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9ca9a51df453dbb11679827f5e962" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42271 } }
I20260811 22:30:57.592821 25928 leader_election.cc:302] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3af9ca9a51df453dbb11679827f5e962; no voters:
I20260811 22:30:57.593163 25928 leader_election.cc:288] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:57.593410 25941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:57.593613 25941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [term 1 LEADER]: Becoming Leader. State: Replica: 3af9ca9a51df453dbb11679827f5e962, State: Running, Role: LEADER
I20260811 22:30:57.593945 25941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9ca9a51df453dbb11679827f5e962" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42271 } }
I20260811 22:30:57.594642 25928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:57.595758 25943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3af9ca9a51df453dbb11679827f5e962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9ca9a51df453dbb11679827f5e962" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42271 } } }
I20260811 22:30:57.595925 25943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:57.596988 25944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3af9ca9a51df453dbb11679827f5e962. Latest consensus state: current_term: 1 leader_uuid: "3af9ca9a51df453dbb11679827f5e962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9ca9a51df453dbb11679827f5e962" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42271 } } }
I20260811 22:30:57.597488 25944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962 [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:57.600049 25969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:57.600116 25969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:57.600206 25962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:57.600998 25962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:57.607000 25962 catalog_manager.cc:1347] Generated new cluster ID: f841dddfa8f1446d9cdd012ad84b8de1
I20260811 22:30:57.607076 25962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:57.620716 25962 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:57.621814 25962 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:57.628855 25962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3af9ca9a51df453dbb11679827f5e962: Generated new TSK 0
I20260811 22:30:57.629598 25962 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
W20260811 22:30:58.449616 19278 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.211.129: real 0.810s user 0.000s sys 0.005s
I20260811 22:30:58.450346 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:42271
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:58.576882 25981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:58.577247 25981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:58.577351 25981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:58.581562 25981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:58.581653 25981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:58.581750 25981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:30:58.587217 25981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:58.588430 25981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:58.589504 25981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:58.600040 25990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:58.600112 25981 server_base.cc:1034] running on GCE node
W20260811 22:30:58.600028 25988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:58.600023 25994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:58.600729 25981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:58.601477 25981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:58.602650 25981 hybrid_clock.cc:648] HybridClock initialized: now 1786487458602632 us; error 42 us; skew 500 ppm
Aug 11 22:30:58 dist-test-slave-8hhm krb5kdc[25765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:58.605824 25981 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:30:58.606837 25981 webserver.cc:460] Webserver started at http://127.18.211.129:45787/ using document root <none> and password file <none>
I20260811 22:30:58.607431 25981 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:58.607525 25981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:58.607771 25981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:58.610083 25981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "f96b700110ee4732899a4d16c6f4f74e"
format_stamp: "Formatted at 2026-08-11 22:30:58 on dist-test-slave-8hhm"
server_key: "92b4c47d049a055f98fd5835f57a5820"
server_key_iv: "32f6455cad337a7462e9fd479c91bbca"
server_key_version: "encryptionkey@0"
I20260811 22:30:58.610677 25981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "f96b700110ee4732899a4d16c6f4f74e"
format_stamp: "Formatted at 2026-08-11 22:30:58 on dist-test-slave-8hhm"
server_key: "92b4c47d049a055f98fd5835f57a5820"
server_key_iv: "32f6455cad337a7462e9fd479c91bbca"
server_key_version: "encryptionkey@0"
I20260811 22:30:58.614703 25981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:58.617336 26005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:58.618637 25981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:30:58.618780 25981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "f96b700110ee4732899a4d16c6f4f74e"
format_stamp: "Formatted at 2026-08-11 22:30:58 on dist-test-slave-8hhm"
server_key: "92b4c47d049a055f98fd5835f57a5820"
server_key_iv: "32f6455cad337a7462e9fd479c91bbca"
server_key_version: "encryptionkey@0"
I20260811 22:30:58.618896 25981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:58.634162 25981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:58.637521 25981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:58.637698 25981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:58.638401 25981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:58.639312 25981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:58.639361 25981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:58.639401 25981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:58.639416 25981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:58.649365 25981 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:45719
I20260811 22:30:58.649487 26219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:45719 every 8 connection(s)
I20260811 22:30:58.650362 25981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:30:58.658999 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25981
I20260811 22:30:58.659157 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:30:58.659503 19278 external_mini_cluster.cc:1428] Setting key b89eee572eb02f75b2d7721fdf50720a
Aug 11 22:30:58 dist-test-slave-8hhm krb5kdc[25765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:58.666100 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:58.652716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:48791 (local address 127.18.211.190:42271)
0811 22:30:58.653013 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:58.653016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:58.653835 (+ 819us) server_negotiation.cc:407] Connection header received
0811 22:30:58.655333 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:58.655336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:58.655399 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:58.655487 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:58.656788 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:58.657442 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:58.658665 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:58.658930 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:58.659052 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:58.662179 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:58.662183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:58.662186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:58.662225 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:58.663849 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:58.664425 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:58.664434 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:58.664435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:58.664516 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:58.664880 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:58.664883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:58.664885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:58.665076 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:58.665189 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:58.665778 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 22:30:58.665913 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":116,"threads_started":1}
I20260811 22:30:58.667004 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:58.653094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:42271 (local address 127.18.211.129:48791)
0811 22:30:58.653647 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:58.653718 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:58.655098 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:58.655668 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:58.655682 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:58.656112 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:58.656621 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:58.656634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:58.657584 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:58.657588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:58.658531 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:58.658541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:58.659102 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:58.659107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:58.659360 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:58.660080 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:58.660103 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:58.661997 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:58.663978 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:58.663985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:58.664001 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:58.664306 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:58.664613 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:58.664617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:58.664619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:58.664774 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:58.665197 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:58.665206 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:58.665515 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 22:30:58.665722 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:30:58.666072 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":183,"threads_started":1}
I20260811 22:30:58.669024 26220 heartbeater.cc:347] Connected to a master server at 127.18.211.190:42271
I20260811 22:30:58.669358 26220 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:58.669894 26220 heartbeater.cc:511] Master 127.18.211.190:42271 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:30:58.671559 25854 ts_manager.cc:194] Registered new tserver with Master: f96b700110ee4732899a4d16c6f4f74e (127.18.211.129:45719)
I20260811 22:30:58.672739 25854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:48791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
W20260811 22:30:59.253178 19278 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.211.130: real 0.592s user 0.002s sys 0.003s
I20260811 22:30:59.253836 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:42271
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:59.371418 26232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:59.371682 26232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:59.371745 26232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:59.375689 26232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:59.375737 26232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:59.375792 26232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:30:59.380872 26232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:59.381949 26232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:59.382905 26232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:59.393203 26247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:59.393206 26244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:59.393231 26245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:59.393585 26232 server_base.cc:1034] running on GCE node
I20260811 22:30:59.394057 26232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:59.394752 26232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:59.395897 26232 hybrid_clock.cc:648] HybridClock initialized: now 1786487459395881 us; error 27 us; skew 500 ppm
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[25765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487459, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:59.398859 26232 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:30:59.399719 26232 webserver.cc:460] Webserver started at http://127.18.211.130:44789/ using document root <none> and password file <none>
I20260811 22:30:59.400298 26232 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:59.400358 26232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:59.400529 26232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:59.402647 26232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "9d170c2e66dd43b485f0946a18f9d46b"
format_stamp: "Formatted at 2026-08-11 22:30:59 on dist-test-slave-8hhm"
server_key: "1ebdace4fa505362d783d8220d85adb0"
server_key_iv: "d6e1242f101ffa365f86eb04354ae51c"
server_key_version: "encryptionkey@0"
I20260811 22:30:59.403178 26232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "9d170c2e66dd43b485f0946a18f9d46b"
format_stamp: "Formatted at 2026-08-11 22:30:59 on dist-test-slave-8hhm"
server_key: "1ebdace4fa505362d783d8220d85adb0"
server_key_iv: "d6e1242f101ffa365f86eb04354ae51c"
server_key_version: "encryptionkey@0"
I20260811 22:30:59.406985 26232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:59.409412 26263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:59.410678 26232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:30:59.410807 26232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "9d170c2e66dd43b485f0946a18f9d46b"
format_stamp: "Formatted at 2026-08-11 22:30:59 on dist-test-slave-8hhm"
server_key: "1ebdace4fa505362d783d8220d85adb0"
server_key_iv: "d6e1242f101ffa365f86eb04354ae51c"
server_key_version: "encryptionkey@0"
I20260811 22:30:59.410912 26232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:59.434661 26232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:59.438481 26232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:59.438733 26232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:59.439419 26232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:59.440346 26232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:59.440420 26232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:59.440490 26232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:59.440550 26232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:59.449282 26232 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:40169
I20260811 22:30:59.449326 26477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:40169 every 8 connection(s)
I20260811 22:30:59.450289 26232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:30:59.452448 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26232
I20260811 22:30:59.452595 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:30:59.452931 19278 external_mini_cluster.cc:1428] Setting key 349786ced07a7948fda9f20827af879a
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[25765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487459, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:30:59.465884 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:59.452215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:44623 (local address 127.18.211.190:42271)
0811 22:30:59.452519 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:59.452524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:59.453658 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 22:30:59.455161 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:59.455164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:59.455217 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:59.455294 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:59.457016 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.457641 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.458839 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.459305 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.459409 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:59.462088 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:59.462091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:59.462093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:59.462118 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:59.463680 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.464206 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.464210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.464212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.464302 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.464653 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.464657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.464659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.464855 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:59.464968 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:59.465545 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 22:30:59.465683 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:30:59.466789 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:59.452652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:42271 (local address 127.18.211.130:44623)
0811 22:30:59.453206 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:59.453290 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:59.454925 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:59.455515 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:59.455528 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:59.456087 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:59.456853 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:59.456882 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.457770 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.457773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:59.458679 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:59.458698 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.459460 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.459464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:59.459647 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:59.460347 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:59.460364 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:59.461900 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:59.463803 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:59.463809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:59.463825 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:59.464089 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:59.464399 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:59.464402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:59.464404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:59.464552 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:59.464966 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:59.464973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:59.465284 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 22:30:59.465487 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:30:59.465830 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":109,"threads_started":1}
I20260811 22:30:59.468061 26478 heartbeater.cc:347] Connected to a master server at 127.18.211.190:42271
I20260811 22:30:59.468322 26478 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:59.468832 26478 heartbeater.cc:511] Master 127.18.211.190:42271 requested a full tablet report, sending...
I20260811 22:30:59.469973 25854 ts_manager.cc:194] Registered new tserver with Master: 9d170c2e66dd43b485f0946a18f9d46b (127.18.211.130:40169)
I20260811 22:30:59.470566 25854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:44623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:30:59.484416 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:42271
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:59.602146 26487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:59.602412 26487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:59.602494 26487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:59.606607 26487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:59.606666 26487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:59.606756 26487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:30:59.612255 26487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:59.613366 26487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:59.614411 26487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:59.624497 26499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:59.624639 26487 server_base.cc:1034] running on GCE node
W20260811 22:30:59.624563 26495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:59.624480 26502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:59.625296 26487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:59.625963 26487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:59.627125 26487 hybrid_clock.cc:648] HybridClock initialized: now 1786487459627102 us; error 53 us; skew 500 ppm
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[25765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487459, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:59.630024 26487 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:30:59.630966 26487 webserver.cc:460] Webserver started at http://127.18.211.131:42139/ using document root <none> and password file <none>
I20260811 22:30:59.631562 26487 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:59.631624 26487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:59.631938 26487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:59.634143 26487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "9354b120bbee4d0493530ac8bfe2de0d"
format_stamp: "Formatted at 2026-08-11 22:30:59 on dist-test-slave-8hhm"
server_key: "b392a6fc94b71fa8bf4ef20125cd9c54"
server_key_iv: "05e0bd9a457d4ecd7a7404918fba84d0"
server_key_version: "encryptionkey@0"
I20260811 22:30:59.634756 26487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "9354b120bbee4d0493530ac8bfe2de0d"
format_stamp: "Formatted at 2026-08-11 22:30:59 on dist-test-slave-8hhm"
server_key: "b392a6fc94b71fa8bf4ef20125cd9c54"
server_key_iv: "05e0bd9a457d4ecd7a7404918fba84d0"
server_key_version: "encryptionkey@0"
I20260811 22:30:59.638643 26487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:59.641115 26514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:59.642416 26487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:30:59.642552 26487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "9354b120bbee4d0493530ac8bfe2de0d"
format_stamp: "Formatted at 2026-08-11 22:30:59 on dist-test-slave-8hhm"
server_key: "b392a6fc94b71fa8bf4ef20125cd9c54"
server_key_iv: "05e0bd9a457d4ecd7a7404918fba84d0"
server_key_version: "encryptionkey@0"
I20260811 22:30:59.642665 26487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:59.657605 26487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:59.660501 26487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:59.660717 26487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:59.661399 26487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:30:59.662259 26487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:30:59.662308 26487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:59.662379 26487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:30:59.662417 26487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:59.671348 26487 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:43253
I20260811 22:30:59.671378 26767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:43253 every 8 connection(s)
I20260811 22:30:59.672298 26487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:30:59.675140 26220 heartbeater.cc:503] Master 127.18.211.190:42271 was elected leader, sending a full tablet report...
I20260811 22:30:59.682245 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26487
I20260811 22:30:59.682376 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[25765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487459, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:59.682672 19278 external_mini_cluster.cc:1428] Setting key 99b88cd6be9d35829564d82b0fe7b67e
I20260811 22:30:59.686909 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:59.674137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:34809 (local address 127.18.211.190:42271)
0811 22:30:59.674282 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:59.674286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:59.675012 (+ 726us) server_negotiation.cc:407] Connection header received
0811 22:30:59.676853 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:59.676856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:59.676908 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:59.677018 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:59.678270 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.678794 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.679937 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.680170 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.680292 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:59.683107 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:59.683110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:59.683112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:59.683136 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:59.684901 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.685467 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.685470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.685471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.685548 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.685867 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.685869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.685870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.686047 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:59.686119 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:59.686652 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 22:30:59.686781 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:30:59.687677 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:59.674438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:42271 (local address 127.18.211.131:34809)
0811 22:30:59.674867 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:59.674910 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:59.676644 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:59.677198 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:59.677205 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:59.677625 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:59.678119 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:59.678132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.678924 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.678927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:59.679807 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:59.679816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.680332 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.680336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:59.680516 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:59.681141 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:59.681158 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:59.682919 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:59.685016 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:59.685021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:59.685033 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:59.685380 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:59.685631 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:59.685634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:59.685635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:59.685778 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:59.686132 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:59.686137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:59.686370 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 22:30:59.686517 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:30:59.686771 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
I20260811 22:30:59.688874 26768 heartbeater.cc:347] Connected to a master server at 127.18.211.190:42271
I20260811 22:30:59.689116 26768 heartbeater.cc:464] Registering TS with master...
I20260811 22:30:59.689630 26768 heartbeater.cc:511] Master 127.18.211.190:42271 requested a full tablet report, sending...
I20260811 22:30:59.690608 25854 ts_manager.cc:194] Registered new tserver with Master: 9354b120bbee4d0493530ac8bfe2de0d (127.18.211.131:43253)
I20260811 22:30:59.691167 25854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:34809
I20260811 22:30:59.696703 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[25765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[25765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:30:59.725170 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:59.715053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.18.211.129:45719)
0811 22:30:59.715351 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:59.715360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:59.715440 (+ 80us) server_negotiation.cc:407] Connection header received
0811 22:30:59.715526 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:59.715535 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:59.715759 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:59.715883 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:59.716575 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.717444 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.718286 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.718685 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.718786 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:59.721269 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:59.721280 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:59.721294 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:59.721321 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:59.723217 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.723679 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.723686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.723688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.723771 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.724050 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.724054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.724056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.724366 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:59.724617 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:59.724831 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 22:30:59.724992 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":125,"threads_started":1}
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[25765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:30:59.735898 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:59.728119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.18.211.190:42271)
0811 22:30:59.728276 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:59.728280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:59.728294 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:30:59.728386 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:59.728388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:59.728427 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:59.728493 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:59.729030 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.729591 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.730336 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:59.730534 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:59.730606 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:59.732828 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:59.732831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:59.732833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:59.732857 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:59.734358 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.734749 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.734751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.734753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.734828 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:59.735059 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:59.735061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:59.735063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:59.735290 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:59.735396 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:59.735586 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 22:30:59.735683 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 22:30:59.736346 25854 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42028
I20260811 22:30:59.737011 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25981
I20260811 22:30:59.757403 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26232
I20260811 22:30:59.782159 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26487
I20260811 22:30:59.800923 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25794
2026-08-11T22:30:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3892 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[26790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[26790](info): set up 2 sockets
Aug 11 22:30:59 dist-test-slave-8hhm krb5kdc[26790](info): commencing operation
krb5kdc: starting...
W20260811 22:31:01.123804 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:01 dist-test-slave-8hhm krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:31:01.255980 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39443
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:39443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:01.373288 26809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:01.373637 26809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:01.373718 26809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:01.377812 26809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:01.377919 26809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:01.377957 26809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:01.377996 26809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:01.378028 26809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:01.384045 26809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:39443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:01.385239 26809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:01.386317 26809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:01.394979 26822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:01.395015 26826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:01.395107 26809 server_base.cc:1034] running on GCE node
W20260811 22:31:01.395045 26831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:01.395841 26809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:01.397038 26809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:01.398221 26809 hybrid_clock.cc:648] HybridClock initialized: now 1786487461398191 us; error 51 us; skew 500 ppm
Aug 11 22:31:01 dist-test-slave-8hhm krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:01.406431 26809 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:31:01.407213 26809 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:01.407284 26809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:01.407465 26809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:01.409698 26809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "66cf603e2e804f44b831e508dfbd857e"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "578fadb69224b77caccae673369fa003"
server_key_iv: "a3f4c774e663206c5d8f87ec5cb329fa"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.410230 26809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "66cf603e2e804f44b831e508dfbd857e"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "578fadb69224b77caccae673369fa003"
server_key_iv: "a3f4c774e663206c5d8f87ec5cb329fa"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.414534 26809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:31:01.417150 26843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:01.418591 26809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:01.418754 26809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "66cf603e2e804f44b831e508dfbd857e"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "578fadb69224b77caccae673369fa003"
server_key_iv: "a3f4c774e663206c5d8f87ec5cb329fa"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.418876 26809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:01.434113 26809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:01.437673 26809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:01.437909 26809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:01.445631 26809 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:39443
I20260811 22:31:01.445922 26937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:39443 every 8 connection(s)
I20260811 22:31:01.446408 26809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:01.449357 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:01.453060 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26809
I20260811 22:31:01.453245 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:01.453510 19278 external_mini_cluster.cc:1428] Setting key 7da5879cb80e9d5686e0cc591cb58a29
I20260811 22:31:01.455624 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e: Bootstrap starting.
I20260811 22:31:01.457971 26938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:01.458753 26938 log.cc:824] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:01.460740 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e: No bootstrap required, opened a new log
Aug 11 22:31:01 dist-test-slave-8hhm krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:01.464445 26938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cf603e2e804f44b831e508dfbd857e" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39443 } }
I20260811 22:31:01.464722 26938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:01.464807 26938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66cf603e2e804f44b831e508dfbd857e, State: Initialized, Role: FOLLOWER
I20260811 22:31:01.465368 26938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cf603e2e804f44b831e508dfbd857e" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39443 } }
I20260811 22:31:01.465535 26938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:01.465631 26938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:01.465749 26938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:01.466990 26938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cf603e2e804f44b831e508dfbd857e" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39443 } }
I20260811 22:31:01.467365 26938 leader_election.cc:302] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66cf603e2e804f44b831e508dfbd857e; no voters:
I20260811 22:31:01.467762 26938 leader_election.cc:288] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:01.467895 26948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:01.468163 26948 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [term 1 LEADER]: Becoming Leader. State: Replica: 66cf603e2e804f44b831e508dfbd857e, State: Running, Role: LEADER
I20260811 22:31:01.468570 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:01.454935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.18.211.190:39443)
0811 22:31:01.455408 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:01.455426 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:01.455463 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:31:01.456172 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:01.456193 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:01.456540 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:01.456903 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:01.457787 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.458859 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.459824 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.460322 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.460506 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:01.463147 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:01.463152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:01.463167 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:01.463197 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:01.465152 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:01.465695 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:01.465702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:01.465708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:01.465868 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:01.466172 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:01.466175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:01.466177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:01.466715 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:01.466950 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:01.467160 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 22:31:01.467474 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":146,"threads_started":1}
I20260811 22:31:01.468552 26948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cf603e2e804f44b831e508dfbd857e" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39443 } }
I20260811 22:31:01.469098 26938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:01.470295 26951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66cf603e2e804f44b831e508dfbd857e. Latest consensus state: current_term: 1 leader_uuid: "66cf603e2e804f44b831e508dfbd857e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cf603e2e804f44b831e508dfbd857e" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39443 } } }
I20260811 22:31:01.470440 26951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:01.470781 26949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66cf603e2e804f44b831e508dfbd857e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cf603e2e804f44b831e508dfbd857e" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39443 } } }
I20260811 22:31:01.470862 26949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e [sys.catalog]: This master's current role is: LEADER
W20260811 22:31:01.474424 26974 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:31:01.474504 26974 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:31:01.474568 26965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:01.475289 26965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:01.480933 26965 catalog_manager.cc:1347] Generated new cluster ID: c88c6dde0aac427fba679469bd2aa65e
I20260811 22:31:01.481015 26965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:01.493942 26965 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:01.494975 26965 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:01.510197 26965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e: Generated new TSK 0
I20260811 22:31:01.511003 26965 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:31:01.545032 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39443
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:01.660712 26985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:01.660995 26985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:01.661041 26985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:01.665088 26985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:01.665156 26985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:01.665278 26985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:01.670855 26985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:01.672039 26985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:01.673113 26985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:01.682778 26998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:01.682778 26993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:01.682778 26992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:01.682832 26985 server_base.cc:1034] running on GCE node
I20260811 22:31:01.683413 26985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:01.684070 26985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:01.685283 26985 hybrid_clock.cc:648] HybridClock initialized: now 1786487461685259 us; error 30 us; skew 500 ppm
Aug 11 22:31:01 dist-test-slave-8hhm krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:01.688248 26985 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:31:01.689050 26985 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:01.689148 26985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:01.689431 26985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:01.691589 26985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "76e0341e8c4f4852a41ad41793db5f47"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "6c05acda9e323417a6352c9e461ca98a"
server_key_iv: "9d74e8e9ffefd40e53bdc349037c02c7"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.692209 26985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "76e0341e8c4f4852a41ad41793db5f47"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "6c05acda9e323417a6352c9e461ca98a"
server_key_iv: "9d74e8e9ffefd40e53bdc349037c02c7"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.696046 26985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:31:01.698591 27005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:01.699860 26985 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:31:01.700001 26985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "76e0341e8c4f4852a41ad41793db5f47"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "6c05acda9e323417a6352c9e461ca98a"
server_key_iv: "9d74e8e9ffefd40e53bdc349037c02c7"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.700119 26985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:01.715431 26985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:01.718950 26985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:01.719168 26985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:01.719795 26985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:01.720676 26985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:01.720758 26985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:01.720831 26985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:01.720883 26985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:01.730163 26985 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:46781
I20260811 22:31:01.730168 27235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:46781 every 8 connection(s)
I20260811 22:31:01.730815 26985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:01.732674 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26985
I20260811 22:31:01.732793 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:01.733114 19278 external_mini_cluster.cc:1428] Setting key 462f86f0b4181e3d8c1f06b46c3683a0
Aug 11 22:31:01 dist-test-slave-8hhm krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:31:01.745653 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:01.732639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:35611 (local address 127.18.211.190:39443)
0811 22:31:01.732805 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:01.732811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:01.733933 (+ 1122us) server_negotiation.cc:407] Connection header received
0811 22:31:01.735363 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:01.735367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:01.735420 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:01.735517 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:01.736853 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.737454 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.738894 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.739119 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.739237 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:01.741891 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:01.741895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:01.741898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:01.741933 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:01.743428 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:01.744020 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:01.744024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:01.744026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:01.744113 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:01.744475 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:01.744480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:01.744482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:01.744732 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:01.744838 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:01.745412 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 22:31:01.745525 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:31:01.746656 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:01.733185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39443 (local address 127.18.211.129:35611)
0811 22:31:01.733788 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:01.733834 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:01.735147 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:01.735697 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:01.735704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:01.736074 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:01.736678 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:01.736701 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.737588 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.737594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:01.738764 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:01.738774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.739281 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.739285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:01.739457 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:01.740124 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:01.740141 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:01.741717 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:01.743556 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:01.743562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:01.743578 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:01.743913 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:01.744215 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:01.744218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:01.744220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:01.744375 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:01.744853 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:01.744859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:01.745115 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 22:31:01.745360 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 22:31:01.745649 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":141,"threads_started":1}
I20260811 22:31:01.748010 27237 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39443
I20260811 22:31:01.748292 27237 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:01.748795 27237 heartbeater.cc:511] Master 127.18.211.190:39443 requested a full tablet report, sending...
I20260811 22:31:01.750451 26871 ts_manager.cc:194] Registered new tserver with Master: 76e0341e8c4f4852a41ad41793db5f47 (127.18.211.129:46781)
I20260811 22:31:01.751598 26871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:35611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:31:01.764359 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39443
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:01.882634 27243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:01.882932 27243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:01.883013 27243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:01.887075 27243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:01.887181 27243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:01.887280 27243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:01.892836 27243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:01.894016 27243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:01.895103 27243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:01.904991 27256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:01.905071 27252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:01.904994 27251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:01.905200 27243 server_base.cc:1034] running on GCE node
I20260811 22:31:01.905905 27243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:01.906564 27243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:01.907765 27243 hybrid_clock.cc:648] HybridClock initialized: now 1786487461907741 us; error 34 us; skew 500 ppm
Aug 11 22:31:01 dist-test-slave-8hhm krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:01.910683 27243 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:31:01.911506 27243 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:01.911612 27243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:01.911864 27243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:01.914121 27243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "87760a522dd54e12bc1d5b6eba7d77da"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "0895beb893666e142e3441f484f8fb32"
server_key_iv: "334cc0d70ec9c0a36c52dff85789d01b"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.914731 27243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "87760a522dd54e12bc1d5b6eba7d77da"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "0895beb893666e142e3441f484f8fb32"
server_key_iv: "334cc0d70ec9c0a36c52dff85789d01b"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.918491 27243 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:31:01.920956 27266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:01.922236 27243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:01.922381 27243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "87760a522dd54e12bc1d5b6eba7d77da"
format_stamp: "Formatted at 2026-08-11 22:31:01 on dist-test-slave-8hhm"
server_key: "0895beb893666e142e3441f484f8fb32"
server_key_iv: "334cc0d70ec9c0a36c52dff85789d01b"
server_key_version: "encryptionkey@0"
I20260811 22:31:01.922502 27243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:01.936479 27243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:01.939963 27243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:01.940191 27243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:01.940819 27243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:01.941787 27243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:01.941861 27243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:01.941936 27243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:01.941982 27243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:01.951318 27243 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:42279
I20260811 22:31:01.951336 27488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:42279 every 8 connection(s)
I20260811 22:31:01.952006 27243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
Aug 11 22:31:01 dist-test-slave-8hhm krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:01.962132 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 27243
I20260811 22:31:01.962297 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:01.962616 19278 external_mini_cluster.cc:1428] Setting key 22bf9492b94c443e041e6bdeaed2d118
I20260811 22:31:01.966828 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:01.953934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:44259 (local address 127.18.211.190:39443)
0811 22:31:01.954091 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:01.954096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:01.954994 (+ 898us) server_negotiation.cc:407] Connection header received
0811 22:31:01.956352 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:01.956355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:01.956414 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:01.956526 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:01.957839 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.958378 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.959575 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.959808 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.959923 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:01.962850 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:01.962853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:01.962855 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:01.962882 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:01.964572 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:01.965142 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:01.965145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:01.965146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:01.965274 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:01.965658 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:01.965662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:01.965664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:01.965860 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:01.965973 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:01.966527 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 22:31:01.966663 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:31:01.968037 27499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:01.954366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39443 (local address 127.18.211.130:44259)
0811 22:31:01.954831 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:01.954875 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:01.956130 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:01.956718 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:01.956726 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:01.957114 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:01.957686 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:01.957700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.958521 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.958524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:01.959427 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:01.959437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:01.959974 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:01.959978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:01.960163 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:01.960755 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:01.960777 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:01.962590 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:01.964700 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:01.964707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:01.964725 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:01.965011 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:01.965377 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:01.965380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:01.965382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:01.965548 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:01.965973 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:01.965982 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:01.966266 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 22:31:01.966471 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 22:31:01.966844 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260811 22:31:01.969506 27491 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39443
I20260811 22:31:01.969776 27491 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:01.970345 27491 heartbeater.cc:511] Master 127.18.211.190:39443 requested a full tablet report, sending...
I20260811 22:31:01.971736 26871 ts_manager.cc:194] Registered new tserver with Master: 87760a522dd54e12bc1d5b6eba7d77da (127.18.211.130:42279)
I20260811 22:31:01.972352 26871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:44259
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:31:01.994195 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39443
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:02.113536 27502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:02.113811 27502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:02.113894 27502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:02.117997 27502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:02.118069 27502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:02.118187 27502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:02.123977 27502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:02.125113 27502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:02.126261 27502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:02.136168 27515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:02.136183 27514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:02.136168 27517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:02.136412 27502 server_base.cc:1034] running on GCE node
I20260811 22:31:02.136996 27502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:02.137758 27502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:02.138955 27502 hybrid_clock.cc:648] HybridClock initialized: now 1786487462138938 us; error 29 us; skew 500 ppm
Aug 11 22:31:02 dist-test-slave-8hhm krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:02.142055 27502 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:31:02.142920 27502 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:02.142989 27502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:02.143237 27502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:02.145516 27502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "f13d992247374cd38201e93a62e626eb"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "0701bcbb28df3bb7e6162a841379a382"
server_key_iv: "946a52d3163f85252e50587e996eed7a"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.146117 27502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "f13d992247374cd38201e93a62e626eb"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "0701bcbb28df3bb7e6162a841379a382"
server_key_iv: "946a52d3163f85252e50587e996eed7a"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.150225 27502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:31:02.152966 27527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:02.154609 27502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:31:02.154754 27502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "f13d992247374cd38201e93a62e626eb"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "0701bcbb28df3bb7e6162a841379a382"
server_key_iv: "946a52d3163f85252e50587e996eed7a"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.154888 27502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:02.169912 27502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:02.173629 27502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:02.173851 27502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:02.174502 27502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:02.175411 27502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:02.175462 27502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:02.175534 27502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:02.175578 27502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:02.185030 27502 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:36863
I20260811 22:31:02.185051 27749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:36863 every 8 connection(s)
I20260811 22:31:02.185729 27502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:02.192517 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 27502
I20260811 22:31:02.192670 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:02.192967 19278 external_mini_cluster.cc:1428] Setting key 2d2b969102f5119dcc3c00ae395389a8
Aug 11 22:31:02 dist-test-slave-8hhm krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:02.199582 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.187567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:56025 (local address 127.18.211.190:39443)
0811 22:31:02.187732 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.187735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.188664 (+ 929us) server_negotiation.cc:407] Connection header received
0811 22:31:02.189966 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.189969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.190018 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.190095 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:02.191231 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.191780 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.193039 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.193390 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.193529 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.196277 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:02.196280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:02.196282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:02.196307 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:02.197692 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:02.198196 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:02.198202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:02.198207 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:02.198308 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:02.198615 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:02.198618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:02.198619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:02.198802 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:02.198879 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.199355 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.199460 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:31:02.200439 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.187951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39443 (local address 127.18.211.131:56025)
0811 22:31:02.188470 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.188541 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:02.189763 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:02.190239 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:02.190245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:02.190606 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:02.191091 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.191104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.191893 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.191896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.192914 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.192924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.193593 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.193599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.193826 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.194495 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:02.194521 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:02.196121 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:02.197796 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:02.197801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:02.197814 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:02.198077 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:02.198393 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:02.198396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:02.198397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:02.198536 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:02.198894 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:02.198899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:02.199141 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 22:31:02.199307 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 22:31:02.199567 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":133,"threads_started":1}
I20260811 22:31:02.201637 27750 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39443
I20260811 22:31:02.201858 27750 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:02.202296 27750 heartbeater.cc:511] Master 127.18.211.190:39443 requested a full tablet report, sending...
I20260811 22:31:02.203293 26871 ts_manager.cc:194] Registered new tserver with Master: f13d992247374cd38201e93a62e626eb (127.18.211.131:36863)
I20260811 22:31:02.203923 26871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:56025
I20260811 22:31:02.207044 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:31:02.216501 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.209029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.18.211.190:39443)
0811 22:31:02.209191 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.209195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.209412 (+ 217us) server_negotiation.cc:407] Connection header received
0811 22:31:02.209572 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.209575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.209625 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.209721 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:02.210480 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.210996 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.211872 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.212095 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.212230 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.213764 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:02.213767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:02.213769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:02.213793 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:02.215039 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:02.215434 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:02.215436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:02.215438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:02.215512 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:02.215749 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:02.215751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:02.215752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:02.215964 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:02.216069 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.216278 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.216379 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:31:02.219609 26871 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:31:02.222764 26871 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:02.238076 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.232409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.18.211.131:36863)
0811 22:31:02.233159 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.233170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.233191 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:31:02.233285 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.233290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.233493 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.233646 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:02.233887 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.234975 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.236457 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.237278 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.237560 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.237669 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.237782 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.237903 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":149,"threads_started":1}
I20260811 22:31:02.238097 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.231720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.18.211.129:46781)
0811 22:31:02.232346 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.232355 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.232654 (+ 299us) server_negotiation.cc:407] Connection header received
0811 22:31:02.232742 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.232749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.232973 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.233134 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:02.233499 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.234709 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.236666 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.237284 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.237417 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.237558 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.237872 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.237961 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":92,"threads_started":1}
I20260811 22:31:02.238101 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.231599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:36863 (local address 127.0.0.1:56774)
0811 22:31:02.232802 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.232820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:02.232916 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:02.233629 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:02.233632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:02.233650 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:02.233761 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.233767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.235106 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.235109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.236350 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.236360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.237412 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.237415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.237571 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.237624 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:31:02.237756 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 22:31:02.237911 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1135,"mutex_wait_us":44,"thread_start_us":151,"threads_started":1}
I20260811 22:31:02.238268 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.231907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:42279 (local address 127.0.0.1:56776)
0811 22:31:02.233296 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.233311 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:02.233443 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:02.233923 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:02.233925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:02.233943 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:02.234022 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.234027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.235020 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.235026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.236247 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.236257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.237398 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.237402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.237981 (+ 579us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.238004 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:31:02.238073 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:31:02.238140 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":26,"thread_start_us":78,"threads_started":1}
I20260811 22:31:02.238430 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.231562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:46781 (local address 127.0.0.1:53282)
0811 22:31:02.232300 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.232352 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:02.232492 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:02.233139 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:02.233145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:02.233184 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:02.233360 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.233367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.234856 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.234860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.236553 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.236560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.237621 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.237624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.237714 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.237751 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:31:02.237817 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:31:02.237911 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":104,"thread_start_us":117,"threads_started":1}
I20260811 22:31:02.238711 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.232563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.18.211.130:42279)
0811 22:31:02.232967 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.232983 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.233421 (+ 438us) server_negotiation.cc:407] Connection header received
0811 22:31:02.233607 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.233614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.233804 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.233951 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:02.234105 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.234870 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.236374 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.237265 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.237405 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.237559 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.238418 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.238535 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":114,"threads_started":1}
I20260811 22:31:02.242057 27104 tablet_service.cc:1467] Processing CreateTablet for tablet a5add482bd2b4e7891c0877bc08d3cdb (DEFAULT_TABLE table=test-table [id=87f0a51189d944db9b42468070772f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:02.242057 27368 tablet_service.cc:1467] Processing CreateTablet for tablet a5add482bd2b4e7891c0877bc08d3cdb (DEFAULT_TABLE table=test-table [id=87f0a51189d944db9b42468070772f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:02.243369 27104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5add482bd2b4e7891c0877bc08d3cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:02.243764 27368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5add482bd2b4e7891c0877bc08d3cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:02.245954 27622 tablet_service.cc:1467] Processing CreateTablet for tablet a5add482bd2b4e7891c0877bc08d3cdb (DEFAULT_TABLE table=test-table [id=87f0a51189d944db9b42468070772f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:02.247179 27622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5add482bd2b4e7891c0877bc08d3cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:02.250460 27779 tablet_bootstrap.cc:492] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Bootstrap starting.
I20260811 22:31:02.251137 27782 tablet_bootstrap.cc:492] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47: Bootstrap starting.
I20260811 22:31:02.253121 27779 tablet_bootstrap.cc:654] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:02.253188 27782 tablet_bootstrap.cc:654] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:02.254216 27782 log.cc:824] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:02.254593 27783 tablet_bootstrap.cc:492] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Bootstrap starting.
I20260811 22:31:02.254643 27779 log.cc:824] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:02.256330 27782 tablet_bootstrap.cc:492] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47: No bootstrap required, opened a new log
I20260811 22:31:02.256560 27782 ts_tablet_manager.cc:1397] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:31:02.256793 27783 tablet_bootstrap.cc:654] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:02.256968 27779 tablet_bootstrap.cc:492] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: No bootstrap required, opened a new log
I20260811 22:31:02.257184 27779 ts_tablet_manager.cc:1397] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:31:02.258190 27783 log.cc:824] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:02.260403 27783 tablet_bootstrap.cc:492] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: No bootstrap required, opened a new log
I20260811 22:31:02.260603 27783 ts_tablet_manager.cc:1397] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:31:02.260792 27782 raft_consensus.cc:348] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.261088 27782 raft_consensus.cc:374] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:02.261145 27782 raft_consensus.cc:729] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e0341e8c4f4852a41ad41793db5f47, State: Initialized, Role: FOLLOWER
I20260811 22:31:02.261420 27779 raft_consensus.cc:348] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.261720 27779 raft_consensus.cc:374] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:02.261780 27779 raft_consensus.cc:729] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87760a522dd54e12bc1d5b6eba7d77da, State: Initialized, Role: FOLLOWER
I20260811 22:31:02.261766 27782 consensus_queue.cc:260] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.262264 27779 consensus_queue.cc:260] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.262506 27237 heartbeater.cc:503] Master 127.18.211.190:39443 was elected leader, sending a full tablet report...
I20260811 22:31:02.262893 27782 ts_tablet_manager.cc:1428] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:31:02.263243 27779 ts_tablet_manager.cc:1428] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:31:02.263635 27491 heartbeater.cc:503] Master 127.18.211.190:39443 was elected leader, sending a full tablet report...
I20260811 22:31:02.264786 27783 raft_consensus.cc:348] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.265087 27783 raft_consensus.cc:374] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:02.265151 27783 raft_consensus.cc:729] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13d992247374cd38201e93a62e626eb, State: Initialized, Role: FOLLOWER
I20260811 22:31:02.265697 27783 consensus_queue.cc:260] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.266399 27750 heartbeater.cc:503] Master 127.18.211.190:39443 was elected leader, sending a full tablet report...
I20260811 22:31:02.266777 27783 ts_tablet_manager.cc:1428] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:31:02.353065 27798 raft_consensus.cc:482] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:02.353416 27798 raft_consensus.cc:504] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.354704 27798 leader_election.cc:288] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76e0341e8c4f4852a41ad41793db5f47 (127.18.211.129:46781), 87760a522dd54e12bc1d5b6eba7d77da (127.18.211.130:42279)
I20260811 22:31:02.359763 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.355103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:55537 (local address 127.18.211.130:42279)
0811 22:31:02.355244 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.355248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.355538 (+ 290us) server_negotiation.cc:407] Connection header received
0811 22:31:02.355605 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.355608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.355657 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.355720 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:02.356001 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.356728 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.358404 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.359013 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.359162 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.359221 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.359584 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.359655 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:31:02.359763 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.355059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:36043 (local address 127.18.211.129:46781)
0811 22:31:02.355179 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.355197 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.355212 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:31:02.355345 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.355349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.355409 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.355477 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:02.355828 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.356640 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.358392 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.359031 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.359143 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.359200 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.359541 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.359620 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:31:02.359771 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.354902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:46781 (local address 127.18.211.131:36043)
0811 22:31:02.355091 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.355108 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:02.355201 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:02.355522 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:02.355525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:02.355538 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:02.355680 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.355691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.356760 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.356763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.358256 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.358267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.359144 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.359148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.359383 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.359426 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:31:02.359509 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:31:02.359592 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":134}
I20260811 22:31:02.359779 27801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.355015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:42279 (local address 127.18.211.131:55537)
0811 22:31:02.355410 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.355421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:02.355533 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:02.355757 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:02.355762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:02.355784 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:02.355892 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.355896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.356840 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.356843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.358256 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:02.358267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.359144 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.359148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:02.359379 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.359430 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:31:02.359558 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:31:02.359605 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":145,"thread_start_us":162,"threads_started":1}
I20260811 22:31:02.360802 27146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5add482bd2b4e7891c0877bc08d3cdb" candidate_uuid: "f13d992247374cd38201e93a62e626eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e0341e8c4f4852a41ad41793db5f47" is_pre_election: true
I20260811 22:31:02.360803 27407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5add482bd2b4e7891c0877bc08d3cdb" candidate_uuid: "f13d992247374cd38201e93a62e626eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87760a522dd54e12bc1d5b6eba7d77da" is_pre_election: true
I20260811 22:31:02.361150 27407 raft_consensus.cc:2454] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13d992247374cd38201e93a62e626eb in term 0.
I20260811 22:31:02.361150 27146 raft_consensus.cc:2454] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13d992247374cd38201e93a62e626eb in term 0.
I20260811 22:31:02.361730 27532 leader_election.cc:302] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 76e0341e8c4f4852a41ad41793db5f47, 87760a522dd54e12bc1d5b6eba7d77da, f13d992247374cd38201e93a62e626eb; no voters:
I20260811 22:31:02.362068 27798 raft_consensus.cc:2781] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:02.362176 27798 raft_consensus.cc:482] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:02.362262 27798 raft_consensus.cc:3037] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:02.363345 27798 raft_consensus.cc:504] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.363710 27798 leader_election.cc:288] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [CANDIDATE]: Term 1 election: Requested vote from peers 76e0341e8c4f4852a41ad41793db5f47 (127.18.211.129:46781), 87760a522dd54e12bc1d5b6eba7d77da (127.18.211.130:42279)
I20260811 22:31:02.364135 27407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5add482bd2b4e7891c0877bc08d3cdb" candidate_uuid: "f13d992247374cd38201e93a62e626eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87760a522dd54e12bc1d5b6eba7d77da"
I20260811 22:31:02.364135 27146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5add482bd2b4e7891c0877bc08d3cdb" candidate_uuid: "f13d992247374cd38201e93a62e626eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e0341e8c4f4852a41ad41793db5f47"
I20260811 22:31:02.364275 27407 raft_consensus.cc:3037] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:02.364275 27146 raft_consensus.cc:3037] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:02.365666 27146 raft_consensus.cc:2454] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13d992247374cd38201e93a62e626eb in term 1.
I20260811 22:31:02.365666 27407 raft_consensus.cc:2454] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13d992247374cd38201e93a62e626eb in term 1.
I20260811 22:31:02.366024 27532 leader_election.cc:302] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87760a522dd54e12bc1d5b6eba7d77da, f13d992247374cd38201e93a62e626eb; no voters:
I20260811 22:31:02.366237 27798 raft_consensus.cc:2781] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:02.366482 27798 raft_consensus.cc:686] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 1 LEADER]: Becoming Leader. State: Replica: f13d992247374cd38201e93a62e626eb, State: Running, Role: LEADER
I20260811 22:31:02.366881 27798 consensus_queue.cc:237] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } }
I20260811 22:31:02.370350 26870 catalog_manager.cc:5495] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb reported cstate change: term changed from 0 to 1, leader changed from <none> to f13d992247374cd38201e93a62e626eb (127.18.211.131). New cstate: current_term: 1 leader_uuid: "f13d992247374cd38201e93a62e626eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f13d992247374cd38201e93a62e626eb" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 } health_report { overall_health: UNKNOWN } } }
I20260811 22:31:02.391072 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.387760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.18.211.131:36863)
0811 22:31:02.387902 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.387905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.387921 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:02.387996 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.387998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.388053 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.388148 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:02.388380 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.388870 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.389741 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:02.389945 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:02.390022 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:02.390319 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:02.390758 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:02.390850 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.390907 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.390956 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:31:02.397522 27407 raft_consensus.cc:1261] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 1 FOLLOWER]: Refusing update from remote peer f13d992247374cd38201e93a62e626eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:02.397522 27146 raft_consensus.cc:1261] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 1 FOLLOWER]: Refusing update from remote peer f13d992247374cd38201e93a62e626eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:02.398449 27802 consensus_queue.cc:1035] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "87760a522dd54e12bc1d5b6eba7d77da" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:02.398682 27798 consensus_queue.cc:1035] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "76e0341e8c4f4852a41ad41793db5f47" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:02.407330 27809 mvcc.cc:205] Tried to move back new op lower bound from 7317452645971857408 to 7317452645857501184. Current Snapshot: MvccSnapshot[applied={T|T < 7317452645971857408}]
I20260811 22:31:02.416790 26870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:02.417073 26870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:02.419890 26870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66cf603e2e804f44b831e508dfbd857e: Sending DeleteTablet for 3 replicas of tablet a5add482bd2b4e7891c0877bc08d3cdb
I20260811 22:31:02.420794 27622 tablet_service.cc:1514] Processing DeleteTablet for tablet a5add482bd2b4e7891c0877bc08d3cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:02 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56774
I20260811 22:31:02.420794 27104 tablet_service.cc:1514] Processing DeleteTablet for tablet a5add482bd2b4e7891c0877bc08d3cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:02 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:53282
I20260811 22:31:02.421046 27368 tablet_service.cc:1514] Processing DeleteTablet for tablet a5add482bd2b4e7891c0877bc08d3cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:02 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:56776
I20260811 22:31:02.421394 27835 tablet_replica.cc:330] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47: stopping tablet replica
I20260811 22:31:02.421577 27838 tablet_replica.cc:330] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: stopping tablet replica
I20260811 22:31:02.421770 27835 raft_consensus.cc:2229] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:02.421972 27838 raft_consensus.cc:2229] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:02.422154 27835 raft_consensus.cc:2258] T a5add482bd2b4e7891c0877bc08d3cdb P 76e0341e8c4f4852a41ad41793db5f47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:02.422165 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26985
I20260811 22:31:02.439942 27837 tablet_replica.cc:330] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: stopping tablet replica
I20260811 22:31:02.440358 27837 raft_consensus.cc:2229] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:02.440842 27837 raft_consensus.cc:2258] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:02.447484 27837 ts_tablet_manager.cc:1905] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:02.450964 27837 ts_tablet_manager.cc:1918] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:02.451083 27837 log.cc:1196] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/a5add482bd2b4e7891c0877bc08d3cdb
I20260811 22:31:02.451408 27837 ts_tablet_manager.cc:1939] T a5add482bd2b4e7891c0877bc08d3cdb P f13d992247374cd38201e93a62e626eb: Deleting consensus metadata
I20260811 22:31:02.452831 26845 catalog_manager.cc:4841] TS f13d992247374cd38201e93a62e626eb (127.18.211.131:36863): tablet a5add482bd2b4e7891c0877bc08d3cdb (table test-table [id=87f0a51189d944db9b42468070772f0a]) successfully deleted
I20260811 22:31:02.462270 27838 raft_consensus.cc:2258] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:02.465662 27838 ts_tablet_manager.cc:1905] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:02.470233 27838 ts_tablet_manager.cc:1918] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:02.470454 27838 log.cc:1196] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/a5add482bd2b4e7891c0877bc08d3cdb
I20260811 22:31:02.470850 27838 ts_tablet_manager.cc:1939] T a5add482bd2b4e7891c0877bc08d3cdb P 87760a522dd54e12bc1d5b6eba7d77da: Deleting consensus metadata
I20260811 22:31:02.472502 26848 catalog_manager.cc:4841] TS 87760a522dd54e12bc1d5b6eba7d77da (127.18.211.130:42279): tablet a5add482bd2b4e7891c0877bc08d3cdb (table test-table [id=87f0a51189d944db9b42468070772f0a]) successfully deleted
W20260811 22:31:02.477165 26846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:46781 (error 108)
I20260811 22:31:02.477402 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 27243
I20260811 22:31:02.488581 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.488165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:46781 (local address 127.0.0.1:53286)
0811 22:31:02.488391 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.488476 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260811 22:31:02.488823 26846 catalog_manager.cc:4584] TS 76e0341e8c4f4852a41ad41793db5f47 (127.18.211.129:46781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5add482bd2b4e7891c0877bc08d3cdb: Network error: Client connection negotiation failed: client connection to 127.18.211.129:46781: connect: Connection refused (error 111)
I20260811 22:31:02.535640 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 27502
I20260811 22:31:02.543861 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.543518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:46781 (local address 127.0.0.1:53292)
0811 22:31:02.543717 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.543769 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 22:31:02.544530 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.544335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:46781 (local address 127.0.0.1:53294)
0811 22:31:02.544440 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:02.544477 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:31:02.555425 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26809
2026-08-11T22:31:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2761 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T22:31:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:02Z Disabled control of system clock
I20260811 22:31:02.602617 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:33875
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:33875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:31:02.718351 27853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:02.718645 27853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:02.718683 27853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:02.722828 27853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:02.722892 27853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:02.722908 27853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:02.722924 27853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:02.722940 27853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:02.727860 27853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:33875
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:33875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:02.728991 27853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:02.730479 27853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:02.739598 27874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:02.739583 27875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:02.739600 27879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:02.739807 27853 server_base.cc:1034] running on GCE node
I20260811 22:31:02.740657 27853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:02.741938 27853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:02.743135 27853 hybrid_clock.cc:648] HybridClock initialized: now 1786487462743115 us; error 32 us; skew 500 ppm
I20260811 22:31:02.745292 27853 webserver.cc:460] Webserver started at http://127.18.211.190:40227/ using document root <none> and password file <none>
I20260811 22:31:02.745923 27853 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:02.746026 27853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:02.746287 27853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:02.748483 27853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "b61756fb43894e53b31299654d56d45f"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "039c0115efa29ae419fd621dc9b0ba52"
server_key_iv: "f24f2ece02623e0c0a21442b914ec975"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.749102 27853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "b61756fb43894e53b31299654d56d45f"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "039c0115efa29ae419fd621dc9b0ba52"
server_key_iv: "f24f2ece02623e0c0a21442b914ec975"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.753104 27853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:31:02.755788 27888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:02.757117 27853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:02.757313 27853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "b61756fb43894e53b31299654d56d45f"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "039c0115efa29ae419fd621dc9b0ba52"
server_key_iv: "f24f2ece02623e0c0a21442b914ec975"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.757450 27853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:02.762423 27853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:02.763118 27853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:02.763332 27853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:02.770805 27990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:33875 every 8 connection(s)
I20260811 22:31:02.770807 27853 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:33875
I20260811 22:31:02.771930 27853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:02.775447 27993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:02.779441 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 27853
I20260811 22:31:02.779597 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:02.779903 19278 external_mini_cluster.cc:1428] Setting key 29b62b3fc588b0ce33d74837e39a9078
I20260811 22:31:02.781885 27993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f: Bootstrap starting.
I20260811 22:31:02.784541 27993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:02.785568 27993 log.cc:824] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:02.786262 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:02.781495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.18.211.190:33875)
0811 22:31:02.782230 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:02.782248 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:02.782285 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:31:02.782985 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:02.783003 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:02.783385 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:02.783700 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:02.784705 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:02.784711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:02.784723 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:02.784735 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:02.784794 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:02.784811 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:02.784902 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:02.785063 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:31:02.785375 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":173,"threads_started":1}
I20260811 22:31:02.787886 27993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f: No bootstrap required, opened a new log
I20260811 22:31:02.791915 27993 raft_consensus.cc:348] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61756fb43894e53b31299654d56d45f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33875 } }
I20260811 22:31:02.792193 27993 raft_consensus.cc:374] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:02.792280 27993 raft_consensus.cc:729] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61756fb43894e53b31299654d56d45f, State: Initialized, Role: FOLLOWER
I20260811 22:31:02.792722 27993 consensus_queue.cc:260] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61756fb43894e53b31299654d56d45f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33875 } }
I20260811 22:31:02.792876 27993 raft_consensus.cc:388] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:02.792943 27993 raft_consensus.cc:482] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:02.793073 27993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:02.794092 27993 raft_consensus.cc:504] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61756fb43894e53b31299654d56d45f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33875 } }
I20260811 22:31:02.794472 27993 leader_election.cc:302] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b61756fb43894e53b31299654d56d45f; no voters:
I20260811 22:31:02.794852 27993 leader_election.cc:288] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:02.795003 28003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:02.795254 28003 raft_consensus.cc:686] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [term 1 LEADER]: Becoming Leader. State: Replica: b61756fb43894e53b31299654d56d45f, State: Running, Role: LEADER
I20260811 22:31:02.795589 28003 consensus_queue.cc:237] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61756fb43894e53b31299654d56d45f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33875 } }
I20260811 22:31:02.796245 27993 sys_catalog.cc:564] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:02.797652 28005 sys_catalog.cc:455] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b61756fb43894e53b31299654d56d45f. Latest consensus state: current_term: 1 leader_uuid: "b61756fb43894e53b31299654d56d45f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61756fb43894e53b31299654d56d45f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33875 } } }
I20260811 22:31:02.797705 28004 sys_catalog.cc:455] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b61756fb43894e53b31299654d56d45f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61756fb43894e53b31299654d56d45f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33875 } } }
I20260811 22:31:02.797796 28005 sys_catalog.cc:458] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:02.797814 28004 sys_catalog.cc:458] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:02.798254 28020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:02.801568 28020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:02.807792 28020 catalog_manager.cc:1347] Generated new cluster ID: d77af017ef904eb99d28fd1fbd60ec6e
I20260811 22:31:02.807880 28020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:02.815155 28020 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:02.815653 28020 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:02.819391 28020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f: Generated new TSK 0
I20260811 22:31:02.820116 28020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:31:02.826661 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33875
--builtin_ntp_servers=127.18.211.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:31:02.949613 28039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:02.949903 28039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:02.949944 28039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:02.954023 28039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:02.954077 28039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:02.954133 28039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:02.958909 28039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:02.960074 28039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:02.961773 28039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:02.971873 28057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:02.971988 28054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:02.972134 28039 server_base.cc:1034] running on GCE node
W20260811 22:31:02.971897 28055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:02.972695 28039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:02.973455 28039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:02.974649 28039 hybrid_clock.cc:648] HybridClock initialized: now 1786487462974627 us; error 32 us; skew 500 ppm
I20260811 22:31:02.976572 28039 webserver.cc:460] Webserver started at http://127.18.211.129:32825/ using document root <none> and password file <none>
I20260811 22:31:02.977254 28039 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:02.977366 28039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:02.977622 28039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:02.979799 28039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "8c6edb1760a943d583c7f6f8963508cc"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "3867f421da030d6852e49cb4dce0873f"
server_key_iv: "2a3b69e067665e1b4cbffb23884368c6"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.980432 28039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "8c6edb1760a943d583c7f6f8963508cc"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "3867f421da030d6852e49cb4dce0873f"
server_key_iv: "2a3b69e067665e1b4cbffb23884368c6"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.984584 28039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:31:02.987349 28069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:02.988617 28039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:02.988771 28039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "8c6edb1760a943d583c7f6f8963508cc"
format_stamp: "Formatted at 2026-08-11 22:31:02 on dist-test-slave-8hhm"
server_key: "3867f421da030d6852e49cb4dce0873f"
server_key_iv: "2a3b69e067665e1b4cbffb23884368c6"
server_key_version: "encryptionkey@0"
I20260811 22:31:02.988904 28039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:02.993752 28039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:02.994477 28039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:02.994673 28039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:02.995253 28039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:02.996224 28039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:02.996299 28039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:02.996371 28039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:02.996423 28039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:03.005686 28039 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:41651
I20260811 22:31:03.005720 28318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:41651 every 8 connection(s)
I20260811 22:31:03.006634 28039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:03.012830 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.008547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:33853 (local address 127.18.211.190:33875)
0811 22:31:03.008687 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.008690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.009765 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 22:31:03.011167 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.011169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.011209 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.011285 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.012448 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.012451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.012452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.012462 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.012476 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.012483 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.012552 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.012684 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.012722 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:31:03.013713 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.008896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33875 (local address 127.18.211.129:33853)
0811 22:31:03.009597 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.009639 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.010973 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.011455 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.011461 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.011550 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.012203 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.012205 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.012247 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.012315 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.012553 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.012560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.012565 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.012697 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.012887 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":104,"threads_started":1}
I20260811 22:31:03.014714 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 28039
I20260811 22:31:03.014853 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:03.015175 19278 external_mini_cluster.cc:1428] Setting key 124dde0bf029274278ceb69ef6caad15
I20260811 22:31:03.015249 28319 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33875
I20260811 22:31:03.015498 28319 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:03.015949 28319 heartbeater.cc:511] Master 127.18.211.190:33875 requested a full tablet report, sending...
I20260811 22:31:03.017381 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33875
--builtin_ntp_servers=127.18.211.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:31:03.017756 27924 ts_manager.cc:194] Registered new tserver with Master: 8c6edb1760a943d583c7f6f8963508cc (127.18.211.129:41651)
W20260811 22:31:03.132217 28326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:03.132470 28326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:03.132515 28326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:03.136369 28326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:03.136418 28326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:03.136471 28326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:03.141109 28326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:03.142297 28326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:03.143715 28326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:03.153810 28339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:03.153718 28343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:03.153741 28340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:03.153925 28326 server_base.cc:1034] running on GCE node
I20260811 22:31:03.154460 28326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:03.155181 28326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:03.156392 28326 hybrid_clock.cc:648] HybridClock initialized: now 1786487463156355 us; error 45 us; skew 500 ppm
I20260811 22:31:03.158463 28326 webserver.cc:460] Webserver started at http://127.18.211.130:35595/ using document root <none> and password file <none>
I20260811 22:31:03.159082 28326 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:03.159147 28326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:03.159391 28326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:03.161681 28326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "2155527d664e4a3c97740829fc57130a"
format_stamp: "Formatted at 2026-08-11 22:31:03 on dist-test-slave-8hhm"
server_key: "ee972ae1f1dd17b45aabf9f94d6c84e9"
server_key_iv: "619882ad02a99e7a1eadc721d78d94f2"
server_key_version: "encryptionkey@0"
I20260811 22:31:03.162335 28326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "2155527d664e4a3c97740829fc57130a"
format_stamp: "Formatted at 2026-08-11 22:31:03 on dist-test-slave-8hhm"
server_key: "ee972ae1f1dd17b45aabf9f94d6c84e9"
server_key_iv: "619882ad02a99e7a1eadc721d78d94f2"
server_key_version: "encryptionkey@0"
I20260811 22:31:03.166189 28326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:31:03.168987 28353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:03.170331 28326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:31:03.170454 28326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "2155527d664e4a3c97740829fc57130a"
format_stamp: "Formatted at 2026-08-11 22:31:03 on dist-test-slave-8hhm"
server_key: "ee972ae1f1dd17b45aabf9f94d6c84e9"
server_key_iv: "619882ad02a99e7a1eadc721d78d94f2"
server_key_version: "encryptionkey@0"
I20260811 22:31:03.170567 28326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:03.175443 28326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:03.176071 28326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:03.176251 28326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:03.176859 28326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:03.178242 28326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:03.178300 28326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:03.178359 28326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:03.178390 28326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:03.187011 28326 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:38127
I20260811 22:31:03.187027 28561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:38127 every 8 connection(s)
I20260811 22:31:03.187947 28326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:03.194438 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.189900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:54381 (local address 127.18.211.190:33875)
0811 22:31:03.190059 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.190064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.191239 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 22:31:03.192703 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.192706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.192745 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.192825 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.194040 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.194042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.194044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.194055 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.194073 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.194081 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.194158 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.194288 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.194325 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:31:03.195124 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 28326
I20260811 22:31:03.195228 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:03.195449 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.190408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33875 (local address 127.18.211.130:54381)
0811 22:31:03.191086 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.191127 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.192483 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.192986 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.192993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.193127 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.193792 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.193793 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.193836 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.193906 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.194160 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.194167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.194172 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.194310 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.194532 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":123,"threads_started":1}
I20260811 22:31:03.195570 19278 external_mini_cluster.cc:1428] Setting key c4bd00cbdbf73d9e7081d3d36746aec3
I20260811 22:31:03.196715 28563 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33875
I20260811 22:31:03.197007 28563 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:03.197578 28563 heartbeater.cc:511] Master 127.18.211.190:33875 requested a full tablet report, sending...
I20260811 22:31:03.197741 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33875
--builtin_ntp_servers=127.18.211.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:31:03.198732 27924 ts_manager.cc:194] Registered new tserver with Master: 2155527d664e4a3c97740829fc57130a (127.18.211.130:38127)
W20260811 22:31:03.312390 28574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:03.312649 28574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:03.312690 28574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:03.317416 28574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:03.317474 28574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:03.317560 28574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:03.322688 28574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:03.323840 28574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:03.325179 28574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:03.334968 28591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:03.334951 28584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:03.335299 28574 server_base.cc:1034] running on GCE node
W20260811 22:31:03.334951 28585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:03.335780 28574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:03.336544 28574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:03.337729 28574 hybrid_clock.cc:648] HybridClock initialized: now 1786487463337714 us; error 26 us; skew 500 ppm
I20260811 22:31:03.339695 28574 webserver.cc:460] Webserver started at http://127.18.211.131:41007/ using document root <none> and password file <none>
I20260811 22:31:03.340298 28574 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:03.340394 28574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:03.340638 28574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:03.342909 28574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4"
format_stamp: "Formatted at 2026-08-11 22:31:03 on dist-test-slave-8hhm"
server_key: "b1fe1232788106d2b03261d18d93a779"
server_key_iv: "223039f9ae9d0a5ca2cb5865543b79a3"
server_key_version: "encryptionkey@0"
I20260811 22:31:03.343495 28574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4"
format_stamp: "Formatted at 2026-08-11 22:31:03 on dist-test-slave-8hhm"
server_key: "b1fe1232788106d2b03261d18d93a779"
server_key_iv: "223039f9ae9d0a5ca2cb5865543b79a3"
server_key_version: "encryptionkey@0"
I20260811 22:31:03.347572 28574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:03.350864 28609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:03.352145 28574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:31:03.352288 28574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4"
format_stamp: "Formatted at 2026-08-11 22:31:03 on dist-test-slave-8hhm"
server_key: "b1fe1232788106d2b03261d18d93a779"
server_key_iv: "223039f9ae9d0a5ca2cb5865543b79a3"
server_key_version: "encryptionkey@0"
I20260811 22:31:03.352409 28574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:03.357096 28574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:03.357734 28574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:03.357924 28574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:03.358464 28574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:03.359393 28574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:03.359467 28574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:03.359537 28574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:03.359588 28574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:03.368934 28574 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:46199
I20260811 22:31:03.368963 28823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:46199 every 8 connection(s)
I20260811 22:31:03.369951 28574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:03.375168 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 28574
I20260811 22:31:03.375300 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:03.375623 19278 external_mini_cluster.cc:1428] Setting key 9bd4381852ab2cf89a184bfba7b98d53
I20260811 22:31:03.376791 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.371961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:60785 (local address 127.18.211.190:33875)
0811 22:31:03.372110 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.372115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.373396 (+ 1281us) server_negotiation.cc:407] Connection header received
0811 22:31:03.374864 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.374868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.374916 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.374999 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.376278 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.376281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.376284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.376296 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.376320 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.376330 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.376415 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.376607 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.376657 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:31:03.377898 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.372469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33875 (local address 127.18.211.131:60785)
0811 22:31:03.373170 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.373294 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.374622 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.375169 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.375178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.375310 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.375973 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.375975 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.376017 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.376085 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.376425 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.376436 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.376444 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.376630 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.376886 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
I20260811 22:31:03.379107 28827 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33875
I20260811 22:31:03.379346 28827 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:03.379787 28827 heartbeater.cc:511] Master 127.18.211.190:33875 requested a full tablet report, sending...
I20260811 22:31:03.380908 27924 ts_manager.cc:194] Registered new tserver with Master: c3d1b29d2ab14440a5d2bd4c446f85c4 (127.18.211.131:46199)
I20260811 22:31:03.389581 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:31:03.393528 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.391461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.18.211.190:33875)
0811 22:31:03.391614 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.391618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.391797 (+ 179us) server_negotiation.cc:407] Connection header received
0811 22:31:03.391893 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.391896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.391961 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.392114 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.392904 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.392908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.392909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.392921 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.392942 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.392950 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.393276 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.393321 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.393354 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:31:03.396420 27924 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:31:03.399762 27924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:03.411679 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.408282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:38127 (local address 127.0.0.1:58598)
0811 22:31:03.408764 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.408811 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.408917 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.409776 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.409783 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.409862 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.410484 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.410486 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.410506 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.410577 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.411297 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.411307 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.411313 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.411454 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.411529 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260811 22:31:03.411726 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.408791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.18.211.130:38127)
0811 22:31:03.409283 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.409297 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.409325 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:31:03.409383 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.409388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.409594 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.410189 (+ 595us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.411089 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.411095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.411106 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.411116 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.411160 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.411194 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.411312 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.411507 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.411580 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":114,"threads_started":1}
I20260811 22:31:03.411968 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.409006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41651 (local address 127.0.0.1:54162)
0811 22:31:03.409698 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.409714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.409780 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.410395 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.410401 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.410421 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.410842 (+ 421us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.410844 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.410854 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.410888 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.411651 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.411654 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.411657 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.411779 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.411824 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":99,"threads_started":1}
I20260811 22:31:03.412222 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.409473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54162 (local address 127.18.211.129:41651)
0811 22:31:03.409933 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.409945 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.409969 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:31:03.410026 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.410032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.410231 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.410663 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.411438 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.411444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.411457 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.411471 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.411511 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.411535 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.411665 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.411926 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.411995 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":124,"threads_started":1}
I20260811 22:31:03.412614 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.408360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:46199 (local address 127.0.0.1:33610)
0811 22:31:03.409254 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.409271 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.409351 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.411085 (+ 1734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.411088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.411102 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.411886 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.411888 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.411898 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.411923 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.412390 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.412393 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.412396 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.412472 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.412505 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":113,"threads_started":1}
I20260811 22:31:03.412974 28853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.408982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.18.211.131:46199)
0811 22:31:03.410022 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.410035 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.410060 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:31:03.410119 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.410126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.410299 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.410422 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.412124 (+ 1702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.412130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.412143 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.412157 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.412203 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.412236 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.412341 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.412771 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.412835 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":164,"threads_started":1}
I20260811 22:31:03.415316 28694 tablet_service.cc:1467] Processing CreateTablet for tablet 6807ec245a0c4eb9b06e547ab32dc9a8 (DEFAULT_TABLE table=test-table [id=68ab101bcb6042b4b8183a6236fc85ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:03.415318 28161 tablet_service.cc:1467] Processing CreateTablet for tablet 6807ec245a0c4eb9b06e547ab32dc9a8 (DEFAULT_TABLE table=test-table [id=68ab101bcb6042b4b8183a6236fc85ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:03.415324 28439 tablet_service.cc:1467] Processing CreateTablet for tablet 6807ec245a0c4eb9b06e547ab32dc9a8 (DEFAULT_TABLE table=test-table [id=68ab101bcb6042b4b8183a6236fc85ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:03.416435 28694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6807ec245a0c4eb9b06e547ab32dc9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:03.416445 28439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6807ec245a0c4eb9b06e547ab32dc9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:03.416445 28161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6807ec245a0c4eb9b06e547ab32dc9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:03.422698 28857 tablet_bootstrap.cc:492] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Bootstrap starting.
I20260811 22:31:03.423966 28860 tablet_bootstrap.cc:492] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Bootstrap starting.
I20260811 22:31:03.424444 28859 tablet_bootstrap.cc:492] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: Bootstrap starting.
I20260811 22:31:03.426261 28857 tablet_bootstrap.cc:654] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:03.426261 28860 tablet_bootstrap.cc:654] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:03.426281 28859 tablet_bootstrap.cc:654] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:03.427032 28859 log.cc:824] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:03.427125 28860 log.cc:824] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:03.427125 28857 log.cc:824] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:03.428965 28859 tablet_bootstrap.cc:492] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: No bootstrap required, opened a new log
I20260811 22:31:03.428966 28857 tablet_bootstrap.cc:492] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: No bootstrap required, opened a new log
I20260811 22:31:03.429190 28859 ts_tablet_manager.cc:1397] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:31:03.429256 28857 ts_tablet_manager.cc:1397] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:31:03.430555 28860 tablet_bootstrap.cc:492] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: No bootstrap required, opened a new log
I20260811 22:31:03.430739 28860 ts_tablet_manager.cc:1397] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:31:03.433073 28859 raft_consensus.cc:348] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.433418 28859 raft_consensus.cc:374] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:03.433478 28859 raft_consensus.cc:729] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c6edb1760a943d583c7f6f8963508cc, State: Initialized, Role: FOLLOWER
I20260811 22:31:03.433859 28857 raft_consensus.cc:348] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.434147 28857 raft_consensus.cc:374] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:03.434206 28857 raft_consensus.cc:729] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d1b29d2ab14440a5d2bd4c446f85c4, State: Initialized, Role: FOLLOWER
I20260811 22:31:03.434186 28859 consensus_queue.cc:260] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.434619 28857 consensus_queue.cc:260] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.434984 28319 heartbeater.cc:503] Master 127.18.211.190:33875 was elected leader, sending a full tablet report...
I20260811 22:31:03.435307 28859 ts_tablet_manager.cc:1428] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:31:03.435343 28860 raft_consensus.cc:348] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.435600 28857 ts_tablet_manager.cc:1428] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:31:03.435601 28860 raft_consensus.cc:374] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:03.435662 28860 raft_consensus.cc:729] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2155527d664e4a3c97740829fc57130a, State: Initialized, Role: FOLLOWER
I20260811 22:31:03.435948 28827 heartbeater.cc:503] Master 127.18.211.190:33875 was elected leader, sending a full tablet report...
I20260811 22:31:03.436103 28860 consensus_queue.cc:260] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.437630 28563 heartbeater.cc:503] Master 127.18.211.190:33875 was elected leader, sending a full tablet report...
I20260811 22:31:03.438454 28860 ts_tablet_manager.cc:1428] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
W20260811 22:31:03.439034 28571 tablet.cc:2369] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:31:03.507910 28320 tablet.cc:2369] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:31:03.621106 28830 tablet.cc:2369] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:31:03.873298 28869 raft_consensus.cc:482] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:03.873551 28869 raft_consensus.cc:504] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.874948 28869 leader_election.cc:288] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d1b29d2ab14440a5d2bd4c446f85c4 (127.18.211.131:46199), 8c6edb1760a943d583c7f6f8963508cc (127.18.211.129:41651)
I20260811 22:31:03.877192 28853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.875300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:47845 (local address 127.18.211.131:46199)
0811 22:31:03.875449 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.875453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.875693 (+ 240us) server_negotiation.cc:407] Connection header received
0811 22:31:03.875861 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.875867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.875915 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.875994 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.876748 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.876752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.876755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.876774 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.876793 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.876801 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.876902 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.876985 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.877027 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:31:03.877238 28875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.875178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:46199 (local address 127.18.211.130:47845)
0811 22:31:03.875569 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.875586 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.875725 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.876047 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.876051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.876067 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.876593 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.876595 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.876604 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.876630 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.876888 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.876893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.876897 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.876984 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.877015 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":128,"threads_started":1}
I20260811 22:31:03.877379 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.875434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:36929 (local address 127.18.211.129:41651)
0811 22:31:03.875560 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.875565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.875808 (+ 243us) server_negotiation.cc:407] Connection header received
0811 22:31:03.875917 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.875920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.875954 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.876057 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.876965 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.876970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.876973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.876985 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.877007 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.877016 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.877106 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.877189 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.877258 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:31:03.877399 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.875296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41651 (local address 127.18.211.130:36929)
0811 22:31:03.875702 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:03.875714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:03.875833 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:03.876110 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:03.876113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:03.876125 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:03.876781 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:03.876782 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:03.876790 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:03.876809 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:03.877107 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:03.877109 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:03.877112 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:31:03.877190 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:31:03.877254 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":69,"threads_started":1}
I20260811 22:31:03.878216 28720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6807ec245a0c4eb9b06e547ab32dc9a8" candidate_uuid: "2155527d664e4a3c97740829fc57130a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" is_pre_election: true
I20260811 22:31:03.878157 28198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6807ec245a0c4eb9b06e547ab32dc9a8" candidate_uuid: "2155527d664e4a3c97740829fc57130a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c6edb1760a943d583c7f6f8963508cc" is_pre_election: true
I20260811 22:31:03.878538 28198 raft_consensus.cc:2454] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2155527d664e4a3c97740829fc57130a in term 0.
I20260811 22:31:03.878543 28720 raft_consensus.cc:2454] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2155527d664e4a3c97740829fc57130a in term 0.
I20260811 22:31:03.879093 28361 leader_election.cc:302] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2155527d664e4a3c97740829fc57130a, 8c6edb1760a943d583c7f6f8963508cc, c3d1b29d2ab14440a5d2bd4c446f85c4; no voters:
I20260811 22:31:03.879364 28869 raft_consensus.cc:2781] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:03.879462 28869 raft_consensus.cc:482] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:03.879521 28869 raft_consensus.cc:3037] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:03.880589 28869 raft_consensus.cc:504] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.880955 28869 leader_election.cc:288] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [CANDIDATE]: Term 1 election: Requested vote from peers c3d1b29d2ab14440a5d2bd4c446f85c4 (127.18.211.131:46199), 8c6edb1760a943d583c7f6f8963508cc (127.18.211.129:41651)
I20260811 22:31:03.881456 28198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6807ec245a0c4eb9b06e547ab32dc9a8" candidate_uuid: "2155527d664e4a3c97740829fc57130a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c6edb1760a943d583c7f6f8963508cc"
I20260811 22:31:03.881480 28720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6807ec245a0c4eb9b06e547ab32dc9a8" candidate_uuid: "2155527d664e4a3c97740829fc57130a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4"
I20260811 22:31:03.881628 28198 raft_consensus.cc:3037] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:03.881626 28720 raft_consensus.cc:3037] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:03.883100 28198 raft_consensus.cc:2454] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2155527d664e4a3c97740829fc57130a in term 1.
I20260811 22:31:03.883100 28720 raft_consensus.cc:2454] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2155527d664e4a3c97740829fc57130a in term 1.
I20260811 22:31:03.883427 28360 leader_election.cc:302] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2155527d664e4a3c97740829fc57130a, 8c6edb1760a943d583c7f6f8963508cc; no voters:
I20260811 22:31:03.883673 28869 raft_consensus.cc:2781] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:03.883898 28869 raft_consensus.cc:686] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 1 LEADER]: Becoming Leader. State: Replica: 2155527d664e4a3c97740829fc57130a, State: Running, Role: LEADER
I20260811 22:31:03.884297 28869 consensus_queue.cc:237] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } }
I20260811 22:31:03.888043 27924 catalog_manager.cc:5495] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2155527d664e4a3c97740829fc57130a (127.18.211.130). New cstate: current_term: 1 leader_uuid: "2155527d664e4a3c97740829fc57130a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2155527d664e4a3c97740829fc57130a" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 38127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 } health_report { overall_health: UNKNOWN } } }
I20260811 22:31:03.975136 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:03.973477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.18.211.130:38127)
0811 22:31:03.973761 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:03.973764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:03.973779 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:31:03.973883 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:03.973885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:03.973952 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:03.974035 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:03.974751 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:03.974756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:03.974760 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:03.974777 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:03.974802 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:03.974813 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:03.974903 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:03.974973 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:31:03.975011 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260811 22:31:03.981622 28198 raft_consensus.cc:1261] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 1 FOLLOWER]: Refusing update from remote peer 2155527d664e4a3c97740829fc57130a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:03.981622 28720 raft_consensus.cc:1261] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 1 FOLLOWER]: Refusing update from remote peer 2155527d664e4a3c97740829fc57130a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:03.982431 28876 consensus_queue.cc:1035] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c6edb1760a943d583c7f6f8963508cc" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:03.982640 28869 consensus_queue.cc:1035] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d1b29d2ab14440a5d2bd4c446f85c4" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:03.991108 28891 mvcc.cc:205] Tried to move back new op lower bound from 7317452652460032000 to 7317452652073152512. Current Snapshot: MvccSnapshot[applied={T|T < 7317452652460032000}]
I20260811 22:31:03.998201 27916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:03.998402 27916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:04.000859 27916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b61756fb43894e53b31299654d56d45f: Sending DeleteTablet for 3 replicas of tablet 6807ec245a0c4eb9b06e547ab32dc9a8
I20260811 22:31:04.001749 28439 tablet_service.cc:1514] Processing DeleteTablet for tablet 6807ec245a0c4eb9b06e547ab32dc9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:03 UTC) from {username='slave'} at 127.0.0.1:58598
I20260811 22:31:04.001721 28694 tablet_service.cc:1514] Processing DeleteTablet for tablet 6807ec245a0c4eb9b06e547ab32dc9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:03 UTC) from {username='slave'} at 127.0.0.1:33610
I20260811 22:31:04.002287 28903 tablet_replica.cc:330] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: stopping tablet replica
I20260811 22:31:04.002275 28161 tablet_service.cc:1514] Processing DeleteTablet for tablet 6807ec245a0c4eb9b06e547ab32dc9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:03 UTC) from {username='slave'} at 127.0.0.1:54162
I20260811 22:31:04.002696 28903 raft_consensus.cc:2229] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:04.002784 28906 tablet_replica.cc:330] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc: stopping tablet replica
I20260811 22:31:04.003093 28906 raft_consensus.cc:2229] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:04.003151 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 28039
I20260811 22:31:04.003458 28906 raft_consensus.cc:2258] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 8c6edb1760a943d583c7f6f8963508cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:04.003482 28903 raft_consensus.cc:2258] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:04.005198 28903 ts_tablet_manager.cc:1905] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:04.005507 28904 tablet_replica.cc:330] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: stopping tablet replica
I20260811 22:31:04.005875 28904 raft_consensus.cc:2229] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:04.006273 28904 raft_consensus.cc:2258] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:04.009274 28903 ts_tablet_manager.cc:1918] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:04.009675 28903 log.cc:1196] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/6807ec245a0c4eb9b06e547ab32dc9a8
I20260811 22:31:04.010803 28904 ts_tablet_manager.cc:1905] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:04.013604 28904 ts_tablet_manager.cc:1918] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:04.014074 28903 ts_tablet_manager.cc:1939] T 6807ec245a0c4eb9b06e547ab32dc9a8 P 2155527d664e4a3c97740829fc57130a: Deleting consensus metadata
I20260811 22:31:04.015522 27894 catalog_manager.cc:4841] TS 2155527d664e4a3c97740829fc57130a (127.18.211.130:38127): tablet 6807ec245a0c4eb9b06e547ab32dc9a8 (table test-table [id=68ab101bcb6042b4b8183a6236fc85ac]) successfully deleted
I20260811 22:31:04.015844 28904 log.cc:1196] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/6807ec245a0c4eb9b06e547ab32dc9a8
I20260811 22:31:04.016261 28904 ts_tablet_manager.cc:1939] T 6807ec245a0c4eb9b06e547ab32dc9a8 P c3d1b29d2ab14440a5d2bd4c446f85c4: Deleting consensus metadata
I20260811 22:31:04.017647 27895 catalog_manager.cc:4841] TS c3d1b29d2ab14440a5d2bd4c446f85c4 (127.18.211.131:46199): tablet 6807ec245a0c4eb9b06e547ab32dc9a8 (table test-table [id=68ab101bcb6042b4b8183a6236fc85ac]) successfully deleted
W20260811 22:31:04.020082 27894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:41651 (error 108)
I20260811 22:31:04.020350 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 28326
I20260811 22:31:04.036312 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:04.033109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41651 (local address 127.0.0.1:54164)
0811 22:31:04.036105 (+ 2996us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:04.036204 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:41651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2920,"thread_start_us":149,"threads_started":1}
W20260811 22:31:04.036536 27894 catalog_manager.cc:4584] TS 8c6edb1760a943d583c7f6f8963508cc (127.18.211.129:41651): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6807ec245a0c4eb9b06e547ab32dc9a8: Network error: Client connection negotiation failed: client connection to 127.18.211.129:41651: connect: Connection refused (error 111)
I20260811 22:31:04.042523 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 28574
I20260811 22:31:04.062071 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 27853
2026-08-11T22:31:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1500 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 22:31:04.093053 19278 mini_oidc.cc:152] Starting JWKS server
I20260811 22:31:04.094918 19278 webserver.cc:460] Webserver started at https://127.0.0.1:40625/ using document root <none> and password file <none>
I20260811 22:31:04.095118 19278 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:31:04.095402 19278 webserver.cc:460] Webserver started at http://127.0.0.1:33079/ using document root <none> and password file <none>
2026-08-11T22:31:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:04Z Disabled control of system clock
I20260811 22:31:04.106205 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34039
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:34039
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40625/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/testchainca.cert with env {}
W20260811 22:31:04.223230 28924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:04.223522 28924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:04.223563 28924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:04.227658 28924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:04.227718 28924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:04.227734 28924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:04.227751 28924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:04.227766 28924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:31:04.227810 28924 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:31:04.227839 28924 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40625/jwks.json
I20260811 22:31:04.233371 28924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:34039
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40625/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:04.234479 28924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:04.235533 28924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:04.244352 28937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:04.244372 28932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:04.244374 28933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:04.244730 28924 server_base.cc:1034] running on GCE node
I20260811 22:31:04.245432 28924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:04.246582 28924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:04.247752 28924 hybrid_clock.cc:648] HybridClock initialized: now 1786487464247740 us; error 31 us; skew 500 ppm
I20260811 22:31:04.249764 28924 webserver.cc:460] Webserver started at http://127.18.211.190:43443/ using document root <none> and password file <none>
I20260811 22:31:04.250389 28924 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:04.250455 28924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:04.250706 28924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:04.252882 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "69c78a15539a42088f9c11c8d0800965"
format_stamp: "Formatted at 2026-08-11 22:31:04 on dist-test-slave-8hhm"
server_key: "6eb9bb6f03f9173faf5526f5157a8cab"
server_key_iv: "df011d219f7c116a2a50a92223786e46"
server_key_version: "encryptionkey@0"
I20260811 22:31:04.253505 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "69c78a15539a42088f9c11c8d0800965"
format_stamp: "Formatted at 2026-08-11 22:31:04 on dist-test-slave-8hhm"
server_key: "6eb9bb6f03f9173faf5526f5157a8cab"
server_key_iv: "df011d219f7c116a2a50a92223786e46"
server_key_version: "encryptionkey@0"
I20260811 22:31:04.257328 28924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:31:04.259887 28945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:04.261147 28924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:04.261327 28924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "69c78a15539a42088f9c11c8d0800965"
format_stamp: "Formatted at 2026-08-11 22:31:04 on dist-test-slave-8hhm"
server_key: "6eb9bb6f03f9173faf5526f5157a8cab"
server_key_iv: "df011d219f7c116a2a50a92223786e46"
server_key_version: "encryptionkey@0"
I20260811 22:31:04.261456 28924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:04.314604 28924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:04.315423 28924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:04.315652 28924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:04.323338 28924 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:34039
I20260811 22:31:04.323380 29052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:34039 every 8 connection(s)
I20260811 22:31:04.324342 28924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:04.327765 29053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:04.333526 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 28924
I20260811 22:31:04.333663 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:04.333683 29053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965: Bootstrap starting.
I20260811 22:31:04.333986 19278 external_mini_cluster.cc:1428] Setting key 4493914529d33d15857f0cdf3f50a681
I20260811 22:31:04.336476 29053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:04.337329 29053 log.cc:824] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:04.339305 29053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965: No bootstrap required, opened a new log
I20260811 22:31:04.343168 29053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c78a15539a42088f9c11c8d0800965" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34039 } }
I20260811 22:31:04.343431 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:04.335317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.18.211.190:34039)
0811 22:31:04.336011 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:04.336026 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:04.336061 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:31:04.336800 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:04.336819 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:04.337335 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:04.337729 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:04.338166 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.339109 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.340079 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.340556 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.340745 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:04.341652 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:04.341658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:04.341671 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:04.342066 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:04.342123 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:04.342144 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:04.342259 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:04.342419 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:31:04.342690 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":163,"threads_started":1}
I20260811 22:31:04.343436 29053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:04.343616 29053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c78a15539a42088f9c11c8d0800965, State: Initialized, Role: FOLLOWER
I20260811 22:31:04.344141 29053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c78a15539a42088f9c11c8d0800965" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34039 } }
I20260811 22:31:04.344310 29053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:04.344365 29053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:04.344452 29053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:04.345469 29053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c78a15539a42088f9c11c8d0800965" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34039 } }
I20260811 22:31:04.345829 29053 leader_election.cc:302] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69c78a15539a42088f9c11c8d0800965; no voters:
I20260811 22:31:04.346244 29053 leader_election.cc:288] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:04.346309 29062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:04.346549 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:31:04.346596 29062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [term 1 LEADER]: Becoming Leader. State: Replica: 69c78a15539a42088f9c11c8d0800965, State: Running, Role: LEADER
I20260811 22:31:04.346966 29062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c78a15539a42088f9c11c8d0800965" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34039 } }
I20260811 22:31:04.347844 29053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:04.348492 29064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69c78a15539a42088f9c11c8d0800965. Latest consensus state: current_term: 1 leader_uuid: "69c78a15539a42088f9c11c8d0800965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c78a15539a42088f9c11c8d0800965" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34039 } } }
I20260811 22:31:04.348645 29064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:04.348978 29070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69c78a15539a42088f9c11c8d0800965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c78a15539a42088f9c11c8d0800965" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34039 } } }
I20260811 22:31:04.349097 29070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:04.349198 29077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:04.352985 29077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:04.359405 29077 catalog_manager.cc:1347] Generated new cluster ID: 615af3425dce432e88e3ace1779c6cae
I20260811 22:31:04.359534 29077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:04.378831 29077 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:04.380090 29077 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:04.394099 29077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69c78a15539a42088f9c11c8d0800965: Generated new TSK 0
I20260811 22:31:04.395115 29077 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:31:04.427776 29114 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 22:31:04.427979 29114 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:31:04.428529 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:04.409863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.18.211.190:34039)
0811 22:31:04.410003 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:04.410007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:04.410291 (+ 284us) server_negotiation.cc:407] Connection header received
0811 22:31:04.410551 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:04.410553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:04.410603 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:04.410702 (+ 99us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:31:04.411063 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.411614 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.413035 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.413329 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.413412 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:04.413874 (+ 462us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:31:04.428119 (+ 14245us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:31:04.428410 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42526: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:31:04.428645 29059 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42526: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 22:31:04.430589 29113 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:04.409670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:34039 (local address 127.0.0.1:42526)
0811 22:31:04.410178 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:04.410199 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:04.410366 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:04.410756 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:04.410760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:04.410771 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:31:04.410927 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:04.410936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.411757 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.411762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:04.412885 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:04.412899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.413467 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.413472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:04.413587 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:04.413657 (+ 70us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:31:04.428451 (+ 14794us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:31:04.428779 (+ 328us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.211.190:34039: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":123,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.211.190:34039: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 22:31:04.431738 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 28924
2026-08-11T22:31:04Z chronyd exiting
I20260811 22:31:04.482358 19278 test_util.cc:175] -----------------------------------------------
I20260811 22:31:04.482472 19278 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487433619239-19278-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (390 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 22:31:04.483696 19278 mini_oidc.cc:152] Starting JWKS server
I20260811 22:31:04.484637 19278 webserver.cc:460] Webserver started at https://127.0.0.1:41785/ using document root <none> and password file <none>
I20260811 22:31:04.484763 19278 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:31:04.485097 19278 webserver.cc:460] Webserver started at http://127.0.0.1:36099/ using document root <none> and password file <none>
2026-08-11T22:31:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:04Z Disabled control of system clock
I20260811 22:31:04.496160 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42065
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:42065
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41785/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/testchainca.cert with env {}
W20260811 22:31:04.611598 29130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:04.611893 29130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:04.611935 29130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:04.615859 29130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:04.615914 29130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:04.615931 29130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:04.615947 29130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:04.615962 29130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:31:04.615978 29130 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:31:04.615998 29130 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41785/jwks.json
I20260811 22:31:04.621486 29130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:42065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41785/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:04.622510 29130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:04.623476 29130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:04.632138 29136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:04.632153 29137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:04.632153 29143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:04.632638 29130 server_base.cc:1034] running on GCE node
I20260811 22:31:04.633188 29130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:04.634384 29130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:04.635584 29130 hybrid_clock.cc:648] HybridClock initialized: now 1786487464635568 us; error 32 us; skew 500 ppm
I20260811 22:31:04.637562 29130 webserver.cc:460] Webserver started at http://127.18.211.190:37391/ using document root <none> and password file <none>
I20260811 22:31:04.638182 29130 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:04.638283 29130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:04.638553 29130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:04.640756 29130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "86240b9d9812460f9843a2539aff80a9"
format_stamp: "Formatted at 2026-08-11 22:31:04 on dist-test-slave-8hhm"
server_key: "eea6255ff5c0590c8f6b44ef757a430e"
server_key_iv: "fd0904b2bdfb6c953329a84b7e4558da"
server_key_version: "encryptionkey@0"
I20260811 22:31:04.641407 29130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "86240b9d9812460f9843a2539aff80a9"
format_stamp: "Formatted at 2026-08-11 22:31:04 on dist-test-slave-8hhm"
server_key: "eea6255ff5c0590c8f6b44ef757a430e"
server_key_iv: "fd0904b2bdfb6c953329a84b7e4558da"
server_key_version: "encryptionkey@0"
I20260811 22:31:04.645395 29130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:31:04.648173 29156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:04.649524 29130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:04.649675 29130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "86240b9d9812460f9843a2539aff80a9"
format_stamp: "Formatted at 2026-08-11 22:31:04 on dist-test-slave-8hhm"
server_key: "eea6255ff5c0590c8f6b44ef757a430e"
server_key_iv: "fd0904b2bdfb6c953329a84b7e4558da"
server_key_version: "encryptionkey@0"
I20260811 22:31:04.649813 29130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:04.663394 29130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:04.664160 29130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:04.664381 29130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:04.671525 29130 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:42065
I20260811 22:31:04.671555 29252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:42065 every 8 connection(s)
I20260811 22:31:04.672586 29130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:04.675797 29253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:04.681639 29253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9: Bootstrap starting.
I20260811 22:31:04.682547 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 29130
I20260811 22:31:04.682706 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:04.682997 19278 external_mini_cluster.cc:1428] Setting key c48c0f75dfea7326a5416ec55f506924
I20260811 22:31:04.684582 29253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:04.685550 29253 log.cc:824] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:04.687626 29253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9: No bootstrap required, opened a new log
I20260811 22:31:04.691459 29253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86240b9d9812460f9843a2539aff80a9" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42065 } }
I20260811 22:31:04.691773 29253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:04.691870 29253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86240b9d9812460f9843a2539aff80a9, State: Initialized, Role: FOLLOWER
I20260811 22:31:04.692389 29253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86240b9d9812460f9843a2539aff80a9" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42065 } }
I20260811 22:31:04.692556 29253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:04.692648 29253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:04.692780 29253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:04.692951 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:04.684145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.18.211.190:42065)
0811 22:31:04.685031 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:04.685066 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:04.685105 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:31:04.685964 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:04.685987 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:04.686381 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:04.686750 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:04.687211 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.688248 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.689190 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.689740 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.689916 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:04.690910 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:04.690915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:04.690929 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:04.691346 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:04.691405 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:04.691426 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:04.691546 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:04.691720 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:31:04.692032 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":146,"threads_started":1}
I20260811 22:31:04.694039 29253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86240b9d9812460f9843a2539aff80a9" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42065 } }
I20260811 22:31:04.694396 29253 leader_election.cc:302] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86240b9d9812460f9843a2539aff80a9; no voters:
I20260811 22:31:04.694787 29253 leader_election.cc:288] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:04.694906 29261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:04.695147 29261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [term 1 LEADER]: Becoming Leader. State: Replica: 86240b9d9812460f9843a2539aff80a9, State: Running, Role: LEADER
I20260811 22:31:04.695474 29261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86240b9d9812460f9843a2539aff80a9" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42065 } }
I20260811 22:31:04.696061 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:31:04.696136 29253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:04.697345 29262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86240b9d9812460f9843a2539aff80a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86240b9d9812460f9843a2539aff80a9" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42065 } } }
I20260811 22:31:04.697489 29262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:04.698005 29264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86240b9d9812460f9843a2539aff80a9. Latest consensus state: current_term: 1 leader_uuid: "86240b9d9812460f9843a2539aff80a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86240b9d9812460f9843a2539aff80a9" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42065 } } }
I20260811 22:31:04.698117 29264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:04.698315 29269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:04.701615 29269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:04.707595 29269 catalog_manager.cc:1347] Generated new cluster ID: 84de4f7672ab4778969e110c5e766cc1
I20260811 22:31:04.707686 29269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:04.732476 29269 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:04.733516 29269 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:04.750459 29269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86240b9d9812460f9843a2539aff80a9: Generated new TSK 0
I20260811 22:31:04.751307 29269 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:31:04.785894 29317 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 22:31:04.786003 29317 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:31:04.786810 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:04.770635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.18.211.190:42065)
0811 22:31:04.770793 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:04.770798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:04.770928 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:31:04.771109 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:04.771111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:04.771163 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:04.771263 (+ 100us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:31:04.771520 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.772083 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.773378 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.773600 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.773687 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:04.774021 (+ 334us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:31:04.786312 (+ 12291us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:31:04.786648 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:31:04.786892 29256 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 22:31:04.786864 29314 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:04.770476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:42065 (local address 127.0.0.1:43910)
0811 22:31:04.770823 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:04.770838 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:04.770962 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:04.771294 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:04.771297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:04.771304 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:31:04.771416 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:04.771421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.772211 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.772214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:04.773268 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:04.773280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:04.773716 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:04.773720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:04.773819 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:04.773851 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:31:04.786542 (+ 12691us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:31:04.786718 (+ 176us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.211.190:42065: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":124,"threads_started":1}
I20260811 22:31:04.787561 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 29130
2026-08-11T22:31:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (368 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 22:31:04.852583 19278 mini_oidc.cc:152] Starting JWKS server
I20260811 22:31:04.853590 19278 webserver.cc:460] Webserver started at https://127.0.0.1:44495/ using document root <none> and password file <none>
I20260811 22:31:04.853732 19278 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:31:04.854012 19278 webserver.cc:460] Webserver started at http://127.0.0.1:44773/ using document root <none> and password file <none>
2026-08-11T22:31:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:04Z Disabled control of system clock
I20260811 22:31:04.864835 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:36365
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:36365
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44495/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:31:04.985589 29331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:04.985904 29331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:04.985945 29331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:04.989946 29331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:04.990012 29331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:04.990027 29331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:04.990044 29331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:04.990059 29331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:31:04.990075 29331 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:31:04.990088 29331 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44495/jwks.json
I20260811 22:31:04.994917 29331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:36365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:36365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44495/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:04.996074 29331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:04.997488 29331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:05.006265 29337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:05.006275 29341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:05.006323 29336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:05.006559 29331 server_base.cc:1034] running on GCE node
I20260811 22:31:05.007232 29331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:05.008335 29331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:05.009500 29331 hybrid_clock.cc:648] HybridClock initialized: now 1786487465009478 us; error 32 us; skew 500 ppm
I20260811 22:31:05.011447 29331 webserver.cc:460] Webserver started at http://127.18.211.190:43695/ using document root <none> and password file <none>
I20260811 22:31:05.012072 29331 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:05.012137 29331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:05.012418 29331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:05.014607 29331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "7b7c4c85fd7f4b43bd33630b6bc20704"
format_stamp: "Formatted at 2026-08-11 22:31:05 on dist-test-slave-8hhm"
server_key: "17c7c81cb0a732812a3052b43fe72f82"
server_key_iv: "6c53b2dae2868942a68e8681289dee18"
server_key_version: "encryptionkey@0"
I20260811 22:31:05.015213 29331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "7b7c4c85fd7f4b43bd33630b6bc20704"
format_stamp: "Formatted at 2026-08-11 22:31:05 on dist-test-slave-8hhm"
server_key: "17c7c81cb0a732812a3052b43fe72f82"
server_key_iv: "6c53b2dae2868942a68e8681289dee18"
server_key_version: "encryptionkey@0"
I20260811 22:31:05.019200 29331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:05.021757 29355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:05.023054 29331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:05.023221 29331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "7b7c4c85fd7f4b43bd33630b6bc20704"
format_stamp: "Formatted at 2026-08-11 22:31:05 on dist-test-slave-8hhm"
server_key: "17c7c81cb0a732812a3052b43fe72f82"
server_key_iv: "6c53b2dae2868942a68e8681289dee18"
server_key_version: "encryptionkey@0"
I20260811 22:31:05.023350 29331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:05.035615 29331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:05.036335 29331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:05.036546 29331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:05.043596 29331 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:36365
I20260811 22:31:05.043627 29439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:36365 every 8 connection(s)
I20260811 22:31:05.044591 29331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:05.047684 29440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:05.051126 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 29331
I20260811 22:31:05.051306 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:05.051592 19278 external_mini_cluster.cc:1428] Setting key 3dede2369a8d18ab001a789e15cd05a8
I20260811 22:31:05.054131 29440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704: Bootstrap starting.
I20260811 22:31:05.057083 29440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:05.058054 29440 log.cc:824] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:05.060410 29440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704: No bootstrap required, opened a new log
I20260811 22:31:05.062232 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:05.053081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.18.211.190:36365)
0811 22:31:05.053898 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:05.053920 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:05.053966 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:31:05.054816 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:05.054843 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:05.055401 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:05.055816 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:05.056287 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.057830 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.058852 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.059349 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.059483 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:05.060365 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:05.060370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:05.060384 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:05.060833 (+ 449us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:05.060888 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:05.060910 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:05.060988 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:05.061162 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:31:05.061514 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":191,"threads_started":1}
I20260811 22:31:05.064260 29440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36365 } }
I20260811 22:31:05.064590 29440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:05.064694 29440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7c4c85fd7f4b43bd33630b6bc20704, State: Initialized, Role: FOLLOWER
I20260811 22:31:05.065126 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:31:05.065254 29440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36365 } }
I20260811 22:31:05.065433 29440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:05.065508 29440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:05.065610 29440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:05.066819 29440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36365 } }
I20260811 22:31:05.067190 29440 leader_election.cc:302] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b7c4c85fd7f4b43bd33630b6bc20704; no voters:
I20260811 22:31:05.067480 29440 leader_election.cc:288] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:05.067600 29447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:05.067862 29447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [term 1 LEADER]: Becoming Leader. State: Replica: 7b7c4c85fd7f4b43bd33630b6bc20704, State: Running, Role: LEADER
I20260811 22:31:05.068185 29447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36365 } }
I20260811 22:31:05.068882 29440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:05.069984 29457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b7c4c85fd7f4b43bd33630b6bc20704. Latest consensus state: current_term: 1 leader_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36365 } } }
I20260811 22:31:05.070128 29457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:05.070482 29452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7c4c85fd7f4b43bd33630b6bc20704" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36365 } } }
I20260811 22:31:05.070547 29470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:05.070608 29452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:05.073508 29470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:05.079006 29470 catalog_manager.cc:1347] Generated new cluster ID: 194e8d5a1f07491a8e0431b1d095a8a4
I20260811 22:31:05.079092 29470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:05.089608 29470 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:05.090521 29470 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:05.101177 29470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b7c4c85fd7f4b43bd33630b6bc20704: Generated new TSK 0
I20260811 22:31:05.102047 29470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:31:05.117630 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:05.112394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53530 (local address 127.18.211.190:36365)
0811 22:31:05.112532 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:05.112535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:05.112666 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:31:05.112846 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:05.112848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:05.112898 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:05.112985 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:31:05.113303 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.113854 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.115111 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.115335 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.115407 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:05.115758 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:31:05.117254 (+ 1496us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:31:05.117487 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53530: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 22:31:05.117705 29445 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53530: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 22:31:05.119537 29495 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:05.112230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:36365 (local address 127.0.0.1:53530)
0811 22:31:05.112568 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:05.112583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:05.112703 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:05.113011 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:05.113014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:05.113023 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:31:05.113149 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:05.113156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.113976 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.113980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:05.114981 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:05.114993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.115459 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.115463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:05.115556 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:05.115585 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:31:05.117426 (+ 1841us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:31:05.119359 (+ 1933us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.211.190:36365: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":1681,"thread_start_us":114,"threads_started":1}
I20260811 22:31:05.120564 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 29331
E20260811 22:31:05.136415 29496 webserver.cc:533] Webserver: sslize failed:
E20260811 22:31:05.136528 29496 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T22:31:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (322 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T22:31:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:05Z Disabled control of system clock
I20260811 22:31:05.185695 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:41231
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:34127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:41231
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/testchainca.cert with env {}
W20260811 22:31:05.302321 29504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:05.302608 29504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:05.302649 29504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:05.306655 29504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:05.306725 29504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:05.306740 29504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:05.306757 29504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:05.306773 29504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:31:05.306790 29504 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:31:05.306804 29504 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 22:31:05.312252 29504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:34127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:41231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:41231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:05.313364 29504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:05.314358 29504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:05.323378 29513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:05.323583 29504 server_base.cc:1034] running on GCE node
W20260811 22:31:05.323381 29512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:05.323379 29524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:05.324368 29504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:05.325567 29504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:05.326766 29504 hybrid_clock.cc:648] HybridClock initialized: now 1786487465326749 us; error 26 us; skew 500 ppm
I20260811 22:31:05.328634 29504 webserver.cc:460] Webserver started at http://127.18.211.190:46829/ using document root <none> and password file <none>
I20260811 22:31:05.329332 29504 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:05.329432 29504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:05.329675 29504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:05.331874 29504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "d7f401ec60254c1f910537124a85e602"
format_stamp: "Formatted at 2026-08-11 22:31:05 on dist-test-slave-8hhm"
server_key: "c0ae13a99cefdcd950f2021a75e61ebe"
server_key_iv: "54cdd6d37c1e6bac33b855951914529e"
server_key_version: "encryptionkey@0"
I20260811 22:31:05.332466 29504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "d7f401ec60254c1f910537124a85e602"
format_stamp: "Formatted at 2026-08-11 22:31:05 on dist-test-slave-8hhm"
server_key: "c0ae13a99cefdcd950f2021a75e61ebe"
server_key_iv: "54cdd6d37c1e6bac33b855951914529e"
server_key_version: "encryptionkey@0"
I20260811 22:31:05.336279 29504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:31:05.338894 29535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:05.340168 29504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:31:05.340341 29504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d7f401ec60254c1f910537124a85e602"
format_stamp: "Formatted at 2026-08-11 22:31:05 on dist-test-slave-8hhm"
server_key: "c0ae13a99cefdcd950f2021a75e61ebe"
server_key_iv: "54cdd6d37c1e6bac33b855951914529e"
server_key_version: "encryptionkey@0"
I20260811 22:31:05.340473 29504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:05.371918 29504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:05.372792 29504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:05.373019 29504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:05.380568 29504 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:41231
I20260811 22:31:05.380596 29624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:41231 every 8 connection(s)
I20260811 22:31:05.381618 29504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:05.382622 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 29504
I20260811 22:31:05.382794 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:05.383098 19278 external_mini_cluster.cc:1428] Setting key ea843983b6c5f6f37ad828305fcc3494
I20260811 22:31:05.385110 29625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:05.391848 29625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602: Bootstrap starting.
I20260811 22:31:05.392477 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:05.384422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.18.211.190:41231)
0811 22:31:05.385064 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:05.385080 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:05.385116 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:31:05.385841 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:05.385859 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:05.386266 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:05.386555 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:05.386900 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.387847 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.389076 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.389603 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.389723 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:05.390663 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:05.390668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:05.390681 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:05.391062 (+ 381us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:05.391116 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:05.391136 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:05.391226 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:05.391387 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:31:05.391692 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":142,"threads_started":1}
I20260811 22:31:05.394359 29625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:05.395251 29625 log.cc:824] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:05.395445 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:31:05.397580 29625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602: No bootstrap required, opened a new log
I20260811 22:31:05.401028 29625 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f401ec60254c1f910537124a85e602" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 41231 } }
I20260811 22:31:05.401352 29625 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:05.401402 29625 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f401ec60254c1f910537124a85e602, State: Initialized, Role: FOLLOWER
I20260811 22:31:05.401840 29625 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f401ec60254c1f910537124a85e602" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 41231 } }
I20260811 22:31:05.401997 29625 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:05.402069 29625 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:05.402220 29625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:05.403363 29625 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f401ec60254c1f910537124a85e602" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 41231 } }
I20260811 22:31:05.403738 29625 leader_election.cc:302] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7f401ec60254c1f910537124a85e602; no voters:
I20260811 22:31:05.404093 29625 leader_election.cc:288] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:05.404242 29633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:05.404512 29633 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [term 1 LEADER]: Becoming Leader. State: Replica: d7f401ec60254c1f910537124a85e602, State: Running, Role: LEADER
I20260811 22:31:05.404887 29633 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f401ec60254c1f910537124a85e602" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 41231 } }
I20260811 22:31:05.405495 29625 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:05.406697 29636 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7f401ec60254c1f910537124a85e602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f401ec60254c1f910537124a85e602" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 41231 } } }
I20260811 22:31:05.406858 29636 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:05.407193 29638 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7f401ec60254c1f910537124a85e602. Latest consensus state: current_term: 1 leader_uuid: "d7f401ec60254c1f910537124a85e602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f401ec60254c1f910537124a85e602" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 41231 } } }
I20260811 22:31:05.407291 29638 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:05.407289 29649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:05.410742 29649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:05.416651 29649 catalog_manager.cc:1347] Generated new cluster ID: 49afa14d09ab4cf9b1997c50d2699dbc
I20260811 22:31:05.416745 29649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:05.442919 29649 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:05.444168 29649 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:05.450129 29649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7f401ec60254c1f910537124a85e602: Generated new TSK 0
I20260811 22:31:05.450881 29649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:31:05.973762 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:05.469656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.18.211.190:41231)
0811 22:31:05.469828 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:05.469834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:05.470007 (+ 173us) server_negotiation.cc:407] Connection header received
0811 22:31:05.470199 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:05.470202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:05.470254 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:05.470353 (+ 99us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:31:05.470635 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.471153 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.472412 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.472619 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.472695 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:05.473047 (+ 352us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:31:05.973315 (+500268us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:31:05.973621 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37478: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:31:05.973874 29630 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37478: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 22:31:05.974354 29675 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:05.469539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:41231 (local address 127.0.0.1:37478)
0811 22:31:05.469901 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:05.469916 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:05.470048 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:05.470379 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:05.470382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:05.470393 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:31:05.470513 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:05.470520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.471278 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.471282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:05.472279 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:05.472292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:05.472742 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:05.472746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:05.472840 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:05.472871 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:31:05.973607 (+500736us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:31:05.973826 (+ 219us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.211.190:41231: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":134,"threads_started":1}
I20260811 22:31:05.975028 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 29504
2026-08-11T22:31:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (826 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (131 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (132 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:31:06 dist-test-slave-8hhm krb5kdc[29699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:31:06 dist-test-slave-8hhm krb5kdc[29699](info): set up 2 sockets
Aug 11 22:31:06 dist-test-slave-8hhm krb5kdc[29699](info): commencing operation
krb5kdc: starting...
W20260811 22:31:07.546168 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:07 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:31:07.680245 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:33345
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:33345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:07.796198 29720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:07.796494 29720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:07.796535 29720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:07.800704 29720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:07.800782 29720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:07.800802 29720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:07.800819 29720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:07.800834 29720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:07.806320 29720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:33345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:33345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:07.807526 29720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:07.808614 29720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:07.817842 29737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:07.818005 29720 server_base.cc:1034] running on GCE node
W20260811 22:31:07.817869 29731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:07.818252 29732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:07.818902 29720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:07.820075 29720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:07.821280 29720 hybrid_clock.cc:648] HybridClock initialized: now 1786487467821260 us; error 34 us; skew 500 ppm
Aug 11 22:31:07 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487467, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:07.829182 29720 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:31:07.830278 29720 webserver.cc:460] Webserver started at http://127.18.211.190:44123/ using document root <none> and password file <none>
I20260811 22:31:07.830904 29720 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:07.831001 29720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:07.831254 29720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:07.833535 29720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "d578c54324344d969891e86411aaf858"
format_stamp: "Formatted at 2026-08-11 22:31:07 on dist-test-slave-8hhm"
server_key: "33920fa4f072edbf9dc3dc60b896feaf"
server_key_iv: "113dfcfaffa5dbfc40a0d8822e37406c"
server_key_version: "encryptionkey@0"
I20260811 22:31:07.834144 29720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "d578c54324344d969891e86411aaf858"
format_stamp: "Formatted at 2026-08-11 22:31:07 on dist-test-slave-8hhm"
server_key: "33920fa4f072edbf9dc3dc60b896feaf"
server_key_iv: "113dfcfaffa5dbfc40a0d8822e37406c"
server_key_version: "encryptionkey@0"
I20260811 22:31:07.838176 29720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:31:07.840797 29750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:07.842171 29720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:07.842324 29720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d578c54324344d969891e86411aaf858"
format_stamp: "Formatted at 2026-08-11 22:31:07 on dist-test-slave-8hhm"
server_key: "33920fa4f072edbf9dc3dc60b896feaf"
server_key_iv: "113dfcfaffa5dbfc40a0d8822e37406c"
server_key_version: "encryptionkey@0"
I20260811 22:31:07.842442 29720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:07.869097 29720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:07.872411 29720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:07.872643 29720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:07.880373 29720 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:33345
I20260811 22:31:07.880399 29844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:33345 every 8 connection(s)
I20260811 22:31:07.881546 29720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:07.884469 29845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:07.888417 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 29720
I20260811 22:31:07.888584 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:07.888929 19278 external_mini_cluster.cc:1428] Setting key 19b8258eda58c795b7e9f64a92bcd485
I20260811 22:31:07.890921 29845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858: Bootstrap starting.
I20260811 22:31:07.893195 29845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:07.894032 29845 log.cc:824] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:07.896251 29845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858: No bootstrap required, opened a new log
Aug 11 22:31:07 dist-test-slave-8hhm krb5kdc[29699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:07.901165 29845 raft_consensus.cc:348] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d578c54324344d969891e86411aaf858" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33345 } }
I20260811 22:31:07.901548 29845 raft_consensus.cc:374] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:07.901616 29845 raft_consensus.cc:729] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d578c54324344d969891e86411aaf858, State: Initialized, Role: FOLLOWER
I20260811 22:31:07.902169 29845 consensus_queue.cc:260] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d578c54324344d969891e86411aaf858" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33345 } }
I20260811 22:31:07.902357 29845 raft_consensus.cc:388] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:07.902429 29845 raft_consensus.cc:482] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:07.902551 29845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:07.903752 29845 raft_consensus.cc:504] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d578c54324344d969891e86411aaf858" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33345 } }
I20260811 22:31:07.904121 29845 leader_election.cc:302] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d578c54324344d969891e86411aaf858; no voters:
I20260811 22:31:07.904455 29845 leader_election.cc:288] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:07.904572 29852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:07.904627 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:07.890481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.18.211.190:33345)
0811 22:31:07.890945 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:07.890961 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:07.891024 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:31:07.891690 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:07.891723 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:07.892058 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:07.892416 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:07.893318 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:07.894412 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:07.895639 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:07.896103 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:07.896232 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:07.899108 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:07.899114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:07.899131 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:07.899164 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:07.901285 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:07.901822 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:07.901828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:07.901834 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:07.901978 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:07.902277 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:07.902280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:07.902282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:07.902837 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:07.903064 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:07.903283 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 22:31:07.903608 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":138,"threads_started":1}
I20260811 22:31:07.904825 29852 raft_consensus.cc:686] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [term 1 LEADER]: Becoming Leader. State: Replica: d578c54324344d969891e86411aaf858, State: Running, Role: LEADER
I20260811 22:31:07.905160 29852 consensus_queue.cc:237] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d578c54324344d969891e86411aaf858" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33345 } }
I20260811 22:31:07.905829 29845 sys_catalog.cc:564] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:07.907405 29855 sys_catalog.cc:455] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d578c54324344d969891e86411aaf858. Latest consensus state: current_term: 1 leader_uuid: "d578c54324344d969891e86411aaf858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d578c54324344d969891e86411aaf858" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33345 } } }
I20260811 22:31:07.907552 29855 sys_catalog.cc:458] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:07.907912 29856 sys_catalog.cc:455] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d578c54324344d969891e86411aaf858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d578c54324344d969891e86411aaf858" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33345 } } }
I20260811 22:31:07.908013 29856 sys_catalog.cc:458] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:07.908354 29861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:07.911522 29861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:07.917367 29861 catalog_manager.cc:1347] Generated new cluster ID: 23d2af23ca884c81a16b3b86578ea15e
I20260811 22:31:07.917447 29861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:07.945574 29861 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:07.946751 29861 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:07.955260 29861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d578c54324344d969891e86411aaf858: Generated new TSK 0
I20260811 22:31:07.956007 29861 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:31:07.992285 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33345
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:08.112351 29895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:08.112632 29895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:08.112677 29895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:08.116637 29895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:08.116706 29895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:08.116767 29895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:08.122133 29895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:08.123131 29895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:08.124120 29895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:08.134509 29920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:08.134625 29915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:08.134528 29916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:08.134997 29895 server_base.cc:1034] running on GCE node
I20260811 22:31:08.135385 29895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:08.135990 29895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:08.137246 29895 hybrid_clock.cc:648] HybridClock initialized: now 1786487468137192 us; error 28 us; skew 500 ppm
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.140164 29895 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:31:08.141130 29895 webserver.cc:460] Webserver started at http://127.18.211.129:33925/ using document root <none> and password file <none>
I20260811 22:31:08.141769 29895 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:08.141877 29895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:08.142136 29895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:08.144261 29895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "c99e3b5c776f480f898c16bb16576e67"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "3ebb1fd08c6073ef6258384bf1dd71a5"
server_key_iv: "60d4319d7d8302f820e65c4a7855704b"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.144867 29895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "c99e3b5c776f480f898c16bb16576e67"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "3ebb1fd08c6073ef6258384bf1dd71a5"
server_key_iv: "60d4319d7d8302f820e65c4a7855704b"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.149129 29895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:31:08.151943 29930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.153355 29895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:08.153501 29895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "c99e3b5c776f480f898c16bb16576e67"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "3ebb1fd08c6073ef6258384bf1dd71a5"
server_key_iv: "60d4319d7d8302f820e65c4a7855704b"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.153635 29895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:08.176115 29895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:08.179363 29895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:08.179594 29895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:08.180310 29895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:08.181197 29895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:08.181294 29895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.181375 29895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:08.181427 29895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.190862 29895 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:42187
I20260811 22:31:08.190891 30141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:42187 every 8 connection(s)
I20260811 22:31:08.191862 29895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:08.200596 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 29895
I20260811 22:31:08.200742 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:08.201129 19278 external_mini_cluster.cc:1428] Setting key 149135faa64a59c548721261dbf75b8f
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:08.207182 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.193752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:46387 (local address 127.18.211.190:33345)
0811 22:31:08.193899 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.193903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.194731 (+ 828us) server_negotiation.cc:407] Connection header received
0811 22:31:08.196165 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.196168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.196227 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.196332 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.197600 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.198147 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.199295 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.199556 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.199685 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.202530 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:08.202534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:08.202544 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:08.202573 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:08.204338 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.204856 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.204862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.204867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.204974 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.205482 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.205490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.205492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.205866 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:08.205983 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:08.206763 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 22:31:08.206926 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:31:08.207949 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.194123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33345 (local address 127.18.211.129:46387)
0811 22:31:08.194583 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:08.194627 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:08.195930 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:08.196521 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:08.196528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:08.196883 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:08.197444 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:08.197460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.198285 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.198289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:08.199164 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:08.199173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.199732 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.199736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:08.199903 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.200492 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:08.200514 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:08.202343 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:08.204463 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:08.204468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:08.204481 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:08.204753 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:08.205087 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:08.205092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:08.205096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:08.205357 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:08.206050 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:08.206057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:08.206414 (+ 357us) client_negotiation.cc:769] Sending connection context
0811 22:31:08.206651 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 22:31:08.206964 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
I20260811 22:31:08.209355 30142 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33345
I20260811 22:31:08.209607 30142 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:08.210124 30142 heartbeater.cc:511] Master 127.18.211.190:33345 requested a full tablet report, sending...
I20260811 22:31:08.211659 29782 ts_manager.cc:194] Registered new tserver with Master: c99e3b5c776f480f898c16bb16576e67 (127.18.211.129:42187)
I20260811 22:31:08.212814 29782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:46387
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:31:08.231868 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33345
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:08.348775 30150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:08.349109 30150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:08.349202 30150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:08.353361 30150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:08.353444 30150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:08.353538 30150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:08.358946 30150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:08.360123 30150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:08.361279 30150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:08.372530 30165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:08.372530 30157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:08.372641 30150 server_base.cc:1034] running on GCE node
W20260811 22:31:08.372531 30158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:08.373282 30150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:08.373981 30150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:08.375147 30150 hybrid_clock.cc:648] HybridClock initialized: now 1786487468375114 us; error 52 us; skew 500 ppm
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.378093 30150 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:31:08.379032 30150 webserver.cc:460] Webserver started at http://127.18.211.130:36771/ using document root <none> and password file <none>
I20260811 22:31:08.379626 30150 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:08.379693 30150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:08.379969 30150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:08.382172 30150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "41764b7c81ab44bb954933c60f90ee6b"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "1bcddc06224b6b67ac7c54a38baaeec2"
server_key_iv: "a833b7df680297abf62cee9b522834f9"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.382782 30150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "41764b7c81ab44bb954933c60f90ee6b"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "1bcddc06224b6b67ac7c54a38baaeec2"
server_key_iv: "a833b7df680297abf62cee9b522834f9"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.386618 30150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:31:08.389119 30182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.390399 30150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:08.390545 30150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "41764b7c81ab44bb954933c60f90ee6b"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "1bcddc06224b6b67ac7c54a38baaeec2"
server_key_iv: "a833b7df680297abf62cee9b522834f9"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.390664 30150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:08.417294 30150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:08.420387 30150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:08.420598 30150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:08.421286 30150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:08.422163 30150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:08.422245 30150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.422319 30150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:08.422371 30150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.431723 30150 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:39165
I20260811 22:31:08.431746 30376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:39165 every 8 connection(s)
I20260811 22:31:08.432690 30150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:08.439469 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 30150
I20260811 22:31:08.439581 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:08.439882 19278 external_mini_cluster.cc:1428] Setting key 31e7f62c0861414d86567e89a180c4e8
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:08.447016 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.434555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:44685 (local address 127.18.211.190:33345)
0811 22:31:08.434695 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.434698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.435562 (+ 864us) server_negotiation.cc:407] Connection header received
0811 22:31:08.436901 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.436903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.436953 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.437030 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.438336 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.438870 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.440145 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.440384 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.440503 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.443075 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:08.443078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:08.443083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:08.443114 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:08.444633 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.445186 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.445192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.445195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.445335 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.445704 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.445708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.445709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.445911 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:08.446025 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:08.446699 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 22:31:08.446838 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:31:08.447911 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.434952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33345 (local address 127.18.211.130:44685)
0811 22:31:08.435400 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:08.435444 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:08.436693 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:08.437292 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:08.437299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:08.437680 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:08.438195 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:08.438210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.438980 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.438983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:08.440014 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:08.440024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.440550 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.440554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:08.440736 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.441399 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:08.441417 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:08.442901 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:08.444770 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:08.444776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:08.444793 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:08.445074 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:08.445435 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:08.445438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:08.445440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:08.445596 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:08.446025 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:08.446033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:08.446408 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 22:31:08.446645 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 22:31:08.446989 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260811 22:31:08.449368 30377 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33345
I20260811 22:31:08.449632 30377 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:08.450162 30377 heartbeater.cc:511] Master 127.18.211.190:33345 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:31:08.451381 29782 ts_manager.cc:194] Registered new tserver with Master: 41764b7c81ab44bb954933c60f90ee6b (127.18.211.130:39165)
I20260811 22:31:08.451993 29782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:44685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:31:08.472256 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33345
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:08.591303 30384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:08.591586 30384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:08.591658 30384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:08.596181 30384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:08.596258 30384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:08.596349 30384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:08.601987 30384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:08.603207 30384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:08.604341 30384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:08.614698 30397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:08.614794 30391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:08.614876 30384 server_base.cc:1034] running on GCE node
W20260811 22:31:08.615162 30394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:08.615796 30384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:08.616501 30384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:08.617694 30384 hybrid_clock.cc:648] HybridClock initialized: now 1786487468617653 us; error 47 us; skew 500 ppm
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.620539 30384 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:31:08.621562 30384 webserver.cc:460] Webserver started at http://127.18.211.131:44705/ using document root <none> and password file <none>
I20260811 22:31:08.622151 30384 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:08.622246 30384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:08.622488 30384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:08.624579 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "df0e45cf8f8a4b01a95279516be208f1"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "2b1736da39f6684f879e6269c5d8ad22"
server_key_iv: "4559e795a895eee787ad040a7faa3390"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.625180 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "df0e45cf8f8a4b01a95279516be208f1"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "2b1736da39f6684f879e6269c5d8ad22"
server_key_iv: "4559e795a895eee787ad040a7faa3390"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.629076 30384 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:31:08.631641 30414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.632915 30384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:08.633054 30384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "df0e45cf8f8a4b01a95279516be208f1"
format_stamp: "Formatted at 2026-08-11 22:31:08 on dist-test-slave-8hhm"
server_key: "2b1736da39f6684f879e6269c5d8ad22"
server_key_iv: "4559e795a895eee787ad040a7faa3390"
server_key_version: "encryptionkey@0"
I20260811 22:31:08.633170 30384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:08.651623 30384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:08.654778 30384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:08.654990 30384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:08.655617 30384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:08.656484 30384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:08.656554 30384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.656623 30384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:08.656672 30384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:08.666239 30384 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:37051
I20260811 22:31:08.666271 30620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:37051 every 8 connection(s)
I20260811 22:31:08.667182 30384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:08.670444 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 30384
I20260811 22:31:08.670569 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:08.670960 19278 external_mini_cluster.cc:1428] Setting key 013d1cf013dc4265adb44843eff28708
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:08.681598 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.669042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:58213 (local address 127.18.211.190:33345)
0811 22:31:08.669258 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.669263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.670138 (+ 875us) server_negotiation.cc:407] Connection header received
0811 22:31:08.672064 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.672070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.672130 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.672205 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.673726 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.674251 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.675350 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.675554 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.675629 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.678202 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:08.678204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:08.678206 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:08.678231 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:08.679720 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.680178 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.680181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.680183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.680259 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.680555 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.680557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.680559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.680745 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:08.680820 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:08.681365 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 22:31:08.681471 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:31:08.682523 30627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.669482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33345 (local address 127.18.211.131:58213)
0811 22:31:08.669971 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:08.670016 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:08.671809 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:08.672448 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:08.672458 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:08.672892 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:08.673582 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:08.673599 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.674360 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.674362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:08.675242 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:08.675252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.675691 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.675694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:08.675842 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.676420 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:08.676437 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:08.678044 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:08.679821 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:08.679826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:08.679838 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:08.680084 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:08.680335 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:08.680337 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:08.680338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:08.680469 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:08.680832 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:08.680837 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:08.681073 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 22:31:08.681291 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 22:31:08.681578 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":129,"threads_started":1}
I20260811 22:31:08.683898 30622 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33345
I20260811 22:31:08.684162 30622 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:08.684859 30622 heartbeater.cc:511] Master 127.18.211.190:33345 requested a full tablet report, sending...
I20260811 22:31:08.685961 29782 ts_manager.cc:194] Registered new tserver with Master: df0e45cf8f8a4b01a95279516be208f1 (127.18.211.131:37051)
I20260811 22:31:08.686833 29782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:58213
I20260811 22:31:08.697310 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/test-admin.keytab.
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.714515 19278 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 22:31:08.717581 30636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:08.716029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.18.211.129:42187)
0811 22:31:08.716354 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.716362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.716393 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:31:08.716466 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.716472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.716690 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.716854 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.717395 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58532: BlockingRecv error: recv got EOF from 127.0.0.1:58532 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
W20260811 22:31:08.717722 19278 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.718796 19278 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.719758 19278 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:31:08.729298 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.720636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.18.211.129:42187)
0811 22:31:08.720797 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.720801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.720876 (+ 75us) server_negotiation.cc:407] Connection header received
0811 22:31:08.720973 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.720975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.721015 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.721081 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.721660 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.722405 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.723208 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.723571 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.723652 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.725995 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:08.726004 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:08.726018 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:08.726042 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:08.727580 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.727993 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.727999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.728000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.728085 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.728339 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.728341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.728342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.728584 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:08.728766 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:08.728973 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 22:31:08.729105 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 22:31:08.733269 30636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:08.732190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.18.211.129:42187)
0811 22:31:08.732311 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.732314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.732439 (+ 125us) server_negotiation.cc:407] Connection header received
0811 22:31:08.732533 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.732535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.732570 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.732630 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.733097 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58554: BlockingRecv error: recv got EOF from 127.0.0.1:58554 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:31:08.733444 19278 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.734439 19278 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.735365 19278 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:31:08.744330 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.736199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.18.211.129:42187)
0811 22:31:08.736324 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.736328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.736444 (+ 116us) server_negotiation.cc:407] Connection header received
0811 22:31:08.736616 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.736619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.736660 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.736754 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.737348 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.737871 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.738726 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.738994 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.739108 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.741375 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:08.741379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:08.741381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:08.741403 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:08.742671 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.743096 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.743100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.743102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.743192 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.743477 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.743481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.743483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.743653 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:08.743812 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:08.744050 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 22:31:08.744160 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 22:31:08.747586 30636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:08.746508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58566 (local address 127.18.211.129:42187)
0811 22:31:08.746641 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.746645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.746740 (+ 95us) server_negotiation.cc:407] Connection header received
0811 22:31:08.746886 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.746889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.746929 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.747022 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.747448 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58566: BlockingRecv error: recv got EOF from 127.0.0.1:58566 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:31:08.747725 19278 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487433619239-19278-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.748904 19278 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:08.749917 19278 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[29699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:31:08.758734 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:08.750762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.18.211.129:42187)
0811 22:31:08.750884 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:08.750888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:08.750970 (+ 82us) server_negotiation.cc:407] Connection header received
0811 22:31:08.751140 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:08.751143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:08.751187 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:08.751272 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:08.751785 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.752288 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.753091 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:08.753365 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:08.753474 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:08.755695 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:08.755698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:08.755700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:08.755722 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:08.757029 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.757481 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.757484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.757485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.757559 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:08.757905 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:08.757907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:08.757908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:08.758066 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:08.758182 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:08.758447 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 22:31:08.758541 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:31:08.759968 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 29895
I20260811 22:31:08.778692 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 30150
I20260811 22:31:08.805686 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 30384
I20260811 22:31:08.820648 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 29720
2026-08-11T22:31:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2586 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[30657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[30657](info): set up 2 sockets
Aug 11 22:31:08 dist-test-slave-8hhm krb5kdc[30657](info): commencing operation
krb5kdc: starting...
W20260811 22:31:10.125849 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:10 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab.
Entry for principal oryx/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab.
I20260811 22:31:10.258613 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39121
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab
--principal=oryx/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:39121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:10.375495 30676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:10.375927 30676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:10.376013 30676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:10.380544 30676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:10.380622 30676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:10.380643 30676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:10.380661 30676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:10.380676 30676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:10.386168 30676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:39121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab
--principal=oryx/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:10.387214 30676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:10.388214 30676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:10.396950 30683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:10.396991 30676 server_base.cc:1034] running on GCE node
W20260811 22:31:10.396950 30686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:10.396945 30684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:10.397922 30676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:10.399032 30676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:10.400229 30676 hybrid_clock.cc:648] HybridClock initialized: now 1786487470400195 us; error 47 us; skew 500 ppm
Aug 11 22:31:10 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:10.403071 30676 init.cc:377] Logged in from keytab as oryx/127.18.211.190@KRBTEST.COM (short username oryx)
I20260811 22:31:10.404062 30676 webserver.cc:460] Webserver started at http://127.18.211.190:39701/ using document root <none> and password file <none>
I20260811 22:31:10.404645 30676 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:10.404757 30676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:10.405007 30676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:10.407244 30676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "545da267b67449a687813d8a1e5680f4"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "e284171297955f11d3113fdfbae41dac"
server_key_iv: "6fc9366d5293953b119b0012b329feb6"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.407855 30676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "545da267b67449a687813d8a1e5680f4"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "e284171297955f11d3113fdfbae41dac"
server_key_iv: "6fc9366d5293953b119b0012b329feb6"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.411933 30676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:31:10.414500 30699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:10.415791 30676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:31:10.415942 30676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "545da267b67449a687813d8a1e5680f4"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "e284171297955f11d3113fdfbae41dac"
server_key_iv: "6fc9366d5293953b119b0012b329feb6"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.416066 30676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:10.425048 30676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:10.428079 30676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:10.428305 30676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:10.435675 30676 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:39121
I20260811 22:31:10.436026 30789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:39121 every 8 connection(s)
I20260811 22:31:10.436781 30676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:10.439749 30797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:10.445823 30797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: Bootstrap starting.
I20260811 22:31:10.446219 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 30676
I20260811 22:31:10.446374 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:10.446662 19278 external_mini_cluster.cc:1428] Setting key c8ae3d38bdbf753bf93b15f590ce3786
I20260811 22:31:10.448629 30797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:10.449492 30797 log.cc:824] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:10.451859 30797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: No bootstrap required, opened a new log
Aug 11 22:31:10 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:10.456189 30797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } }
I20260811 22:31:10.456527 30797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:10.456614 30797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545da267b67449a687813d8a1e5680f4, State: Initialized, Role: FOLLOWER
I20260811 22:31:10.457185 30797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } }
I20260811 22:31:10.457410 30797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:10.457507 30797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:10.457638 30797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:10.459026 30797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } }
I20260811 22:31:10.459444 30797 leader_election.cc:302] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545da267b67449a687813d8a1e5680f4; no voters:
I20260811 22:31:10.459864 30797 leader_election.cc:288] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:10.459977 30805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:10.460249 30805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 1 LEADER]: Becoming Leader. State: Replica: 545da267b67449a687813d8a1e5680f4, State: Running, Role: LEADER
I20260811 22:31:10.460598 30805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } }
I20260811 22:31:10.461411 30797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:10.462355 30807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545da267b67449a687813d8a1e5680f4. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } }
I20260811 22:31:10.462509 30807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:10.462829 30806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } }
I20260811 22:31:10.462929 30806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:10.463238 30815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:10.463706 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:10.448148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.18.211.190:39121)
0811 22:31:10.448577 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:10.448594 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:10.448630 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:31:10.449466 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:10.449486 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:10.449881 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:10.450169 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:10.451009 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:10.452467 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:10.453413 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:10.454143 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:10.454318 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:10.456906 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:10.456912 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:10.456930 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:10.456965 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:10.459297 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:10.459806 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:10.459815 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:10.459822 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:10.459982 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:10.460375 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:10.460378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:10.460380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:10.460906 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:10.461935 (+ 1029us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:10.462099 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 22:31:10.462487 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":141,"threads_started":1}
I20260811 22:31:10.466742 30815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:10.473147 30815 catalog_manager.cc:1347] Generated new cluster ID: 72d2d6dc099a40d1bc2fab835e24ead2
I20260811 22:31:10.473251 30815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:10.498308 30815 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:10.499353 30815 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:10.508514 30815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: Generated new TSK 0
I20260811 22:31:10.509361 30815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab.
Entry for principal oryx/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab.
I20260811 22:31:10.545022 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39121
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab
--principal=oryx/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:10.661927 30838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:10.662293 30838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:10.662382 30838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:10.666719 30838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:10.666787 30838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:10.666847 30838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:10.672199 30838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab
--principal=oryx/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:10.673280 30838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:10.674297 30838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:10.684011 30848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:10.684020 30847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:10.684020 30850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:10.684284 30838 server_base.cc:1034] running on GCE node
I20260811 22:31:10.684684 30838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:10.685367 30838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:10.686530 30838 hybrid_clock.cc:648] HybridClock initialized: now 1786487470686492 us; error 49 us; skew 500 ppm
Aug 11 22:31:10 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:10.689347 30838 init.cc:377] Logged in from keytab as oryx/127.18.211.129@KRBTEST.COM (short username oryx)
I20260811 22:31:10.690261 30838 webserver.cc:460] Webserver started at http://127.18.211.129:36799/ using document root <none> and password file <none>
I20260811 22:31:10.690856 30838 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:10.690956 30838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:10.691219 30838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:10.693446 30838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "e3cdc0758af94b74b610e3093a7b8a5a"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "049f542f7879281404a2b22244fa9d6b"
server_key_iv: "f91e2e3c427a609d46f31ca5314c9619"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.694037 30838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "e3cdc0758af94b74b610e3093a7b8a5a"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "049f542f7879281404a2b22244fa9d6b"
server_key_iv: "f91e2e3c427a609d46f31ca5314c9619"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.697906 30838 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:31:10.700407 30862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:10.701642 30838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:10.701789 30838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "e3cdc0758af94b74b610e3093a7b8a5a"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "049f542f7879281404a2b22244fa9d6b"
server_key_iv: "f91e2e3c427a609d46f31ca5314c9619"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.701905 30838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:10.746353 30838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:10.749461 30838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:10.749694 30838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:10.750360 30838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:10.751305 30838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:10.751394 30838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:10.751468 30838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:10.751523 30838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:10.760844 30838 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:38915
I20260811 22:31:10.760869 31066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:38915 every 8 connection(s)
I20260811 22:31:10.761883 30838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:10.763271 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 30838
I20260811 22:31:10.763440 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:10.763773 19278 external_mini_cluster.cc:1428] Setting key 2eb57e055253023e2e8898086ed0b741
Aug 11 22:31:10 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:31:10.776983 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:10.764192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:46235 (local address 127.18.211.190:39121)
0811 22:31:10.764335 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:10.764340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:10.765296 (+ 956us) server_negotiation.cc:407] Connection header received
0811 22:31:10.766729 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:10.766735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:10.766803 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:10.766900 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:10.768125 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:10.768658 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:10.769931 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:10.770176 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:10.770293 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:10.773270 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:10.773277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:10.773281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:10.773312 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:10.774978 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:10.775524 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:10.775527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:10.775528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:10.775624 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:10.775961 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:10.775964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:10.775966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:10.776189 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:10.776272 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:10.776752 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 22:31:10.776856 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.130@KRBTEST.COM" created.
I20260811 22:31:10.778069 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:10.764613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.129:46235)
0811 22:31:10.765112 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:10.765167 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:10.766508 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:10.767087 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:10.767094 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:10.767476 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:10.767964 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:10.767978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:10.768798 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:10.768802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:10.769796 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:10.769807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:10.770334 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:10.770339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:10.770520 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:10.771148 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:10.771166 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:10.773072 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:10.775138 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:10.775143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:10.775155 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:10.775428 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:10.775711 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:10.775713 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:10.775715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:10.775850 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:10.776293 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:10.776299 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:10.776538 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 22:31:10.776705 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 22:31:10.776966 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":148,"threads_started":1}
I20260811 22:31:10.779553 31067 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39121
I20260811 22:31:10.779879 31067 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:10.780540 31067 heartbeater.cc:511] Master 127.18.211.190:39121 requested a full tablet report, sending...
I20260811 22:31:10.782294 30730 ts_manager.cc:194] Registered new tserver with Master: e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:10.783443 30730 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:46235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab.
Entry for principal oryx/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab.
I20260811 22:31:10.796617 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39121
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab
--principal=oryx/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:10.917784 31072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:10.918064 31072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:10.918108 31072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:10.922278 31072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:10.922351 31072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:10.922444 31072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:10.927812 31072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab
--principal=oryx/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:10.928922 31072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:10.930023 31072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:10.940482 31080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:10.940516 31079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:10.940512 31085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:10.940876 31072 server_base.cc:1034] running on GCE node
I20260811 22:31:10.941416 31072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:10.942114 31072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:10.943341 31072 hybrid_clock.cc:648] HybridClock initialized: now 1786487470943321 us; error 39 us; skew 500 ppm
Aug 11 22:31:10 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:10.946403 31072 init.cc:377] Logged in from keytab as oryx/127.18.211.130@KRBTEST.COM (short username oryx)
I20260811 22:31:10.947402 31072 webserver.cc:460] Webserver started at http://127.18.211.130:40869/ using document root <none> and password file <none>
I20260811 22:31:10.948058 31072 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:10.948168 31072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:10.948436 31072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:10.950843 31072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "93a01bbb5e8f4acdbdd519dcc69ca548"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "d8d890a615eeb38ac8ec6e546e589d5c"
server_key_iv: "52b9cb29955aba8db16e4d2d3821029c"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.951495 31072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "93a01bbb5e8f4acdbdd519dcc69ca548"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "d8d890a615eeb38ac8ec6e546e589d5c"
server_key_iv: "52b9cb29955aba8db16e4d2d3821029c"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.955792 31072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 22:31:10.958696 31094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:10.960094 31072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:31:10.960281 31072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "93a01bbb5e8f4acdbdd519dcc69ca548"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "d8d890a615eeb38ac8ec6e546e589d5c"
server_key_iv: "52b9cb29955aba8db16e4d2d3821029c"
server_key_version: "encryptionkey@0"
I20260811 22:31:10.960430 31072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:10.979308 31072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:10.982702 31072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:10.982946 31072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:10.983640 31072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:10.984591 31072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:10.984671 31072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:10.984753 31072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:10.984817 31072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:10.994580 31072 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:35859
I20260811 22:31:10.994597 31282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:35859 every 8 connection(s)
I20260811 22:31:10.995617 31072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:11.005574 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 31072
I20260811 22:31:11.005708 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:11.006011 19278 external_mini_cluster.cc:1428] Setting key f2f2ba8c3fc499a0e2c6447e4472b776
I20260811 22:31:11.010449 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:10.997665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:53195 (local address 127.18.211.190:39121)
0811 22:31:10.997818 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:10.997822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:10.998696 (+ 874us) server_negotiation.cc:407] Connection header received
0811 22:31:11.000096 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.000099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.000149 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.000231 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.001444 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.001971 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.003088 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.003301 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.003385 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.006206 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.006210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.006250 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.006282 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.007972 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.008650 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.008656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.008659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.008769 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.009142 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.009145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.009147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.009388 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.009496 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.010179 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.010300 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:11.011238 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:10.998057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.130:53195)
0811 22:31:10.998529 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:10.998573 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:10.999837 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.000405 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.000412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.000778 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.001297 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.001312 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.002087 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.002089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.002967 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.002976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.003438 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.003441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.003581 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.004239 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.004256 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.006020 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.008133 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.008140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.008156 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.008535 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.008866 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.008870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.008872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.009023 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.009506 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.009523 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.009773 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.009965 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.010307 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":131,"threads_started":1}
I20260811 22:31:11.012593 31283 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39121
I20260811 22:31:11.012887 31283 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:11.013485 31283 heartbeater.cc:511] Master 127.18.211.190:39121 requested a full tablet report, sending...
I20260811 22:31:11.014678 30730 ts_manager.cc:194] Registered new tserver with Master: 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130:35859)
I20260811 22:31:11.015358 30730 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:53195
WARNING: no policy specified for oryx/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab.
Entry for principal oryx/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab.
I20260811 22:31:11.036478 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39121
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab
--principal=oryx/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:11.154649 31290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:11.154990 31290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:11.155102 31290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:11.159755 31290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:11.159876 31290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:11.159992 31290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:11.166517 31290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab
--principal=oryx/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:11.167757 31290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:11.168910 31290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:11.180018 31303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:11.180022 31308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:11.180075 31290 server_base.cc:1034] running on GCE node
W20260811 22:31:11.180018 31302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:11.180819 31290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:11.181663 31290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:11.182873 31290 hybrid_clock.cc:648] HybridClock initialized: now 1786487471182810 us; error 70 us; skew 500 ppm
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:11.186122 31290 init.cc:377] Logged in from keytab as oryx/127.18.211.131@KRBTEST.COM (short username oryx)
I20260811 22:31:11.187075 31290 webserver.cc:460] Webserver started at http://127.18.211.131:41503/ using document root <none> and password file <none>
I20260811 22:31:11.187652 31290 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:11.187718 31290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:11.187978 31290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:11.190385 31290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "c4361fe589634297aeca20acc858faf0"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "86c4a9dffcf7fdd66ee34d170e95012a"
server_key_iv: "3d968fea3e0e8c6857626398dbfe477d"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.191032 31290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "c4361fe589634297aeca20acc858faf0"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "86c4a9dffcf7fdd66ee34d170e95012a"
server_key_iv: "3d968fea3e0e8c6857626398dbfe477d"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.195237 31290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:31:11.197837 31323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.199203 31290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:11.199365 31290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "c4361fe589634297aeca20acc858faf0"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "86c4a9dffcf7fdd66ee34d170e95012a"
server_key_iv: "3d968fea3e0e8c6857626398dbfe477d"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.199507 31290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:11.226907 31290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:11.230259 31290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:11.230499 31290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:11.231180 31290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:11.232127 31290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:11.232208 31290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.232290 31290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:11.232338 31290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.242471 31290 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:42535
I20260811 22:31:11.242494 31522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:42535 every 8 connection(s)
I20260811 22:31:11.243521 31290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:11.244549 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 31290
I20260811 22:31:11.244697 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:11.245030 19278 external_mini_cluster.cc:1428] Setting key acee83f5d6ddd7fc44c9673d24bf2b00
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:11.258505 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.246144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:41767 (local address 127.18.211.190:39121)
0811 22:31:11.246297 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.246303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.247348 (+ 1045us) server_negotiation.cc:407] Connection header received
0811 22:31:11.248837 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.248840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.248885 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.248954 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.250205 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.250711 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.251821 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.252024 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.252115 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.254794 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.254797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.254799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.254826 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.256351 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.256864 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.256867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.256868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.256945 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.257347 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.257349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.257350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.257539 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.257619 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.258162 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.258331 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:31:11.259537 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.246683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:41767)
0811 22:31:11.247197 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.247252 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.248636 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.249134 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.249141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.249582 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.250078 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.250092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.250818 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.250820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.251717 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.251727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.252153 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.252155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.252308 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.252965 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.252982 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.254610 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.256469 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.256474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.256487 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.256774 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.257023 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.257025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.257027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.257187 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.257631 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.257637 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.257888 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.258114 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.258472 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":149,"threads_started":1}
I20260811 22:31:11.260670 31524 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39121
I20260811 22:31:11.260943 31524 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:11.261575 31524 heartbeater.cc:511] Master 127.18.211.190:39121 requested a full tablet report, sending...
I20260811 22:31:11.262722 30730 ts_manager.cc:194] Registered new tserver with Master: c4361fe589634297aeca20acc858faf0 (127.18.211.131:42535)
I20260811 22:31:11.263375 30730 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:41767
I20260811 22:31:11.271801 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.211.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.189.keytab.
Entry for principal oryx/127.18.211.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.189.keytab.
I20260811 22:31:11.300920 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:39809
--webserver_interface=127.18.211.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.189.keytab
--principal=oryx/127.18.211.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:39121,127.18.211.189:39809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:11.423750 31532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:11.424001 31532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:11.424041 31532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:11.427985 31532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:11.428041 31532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:11.428061 31532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:11.428077 31532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:11.428118 31532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:11.433573 31532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:39121,127.18.211.189:39809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.189.keytab
--principal=oryx/127.18.211.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:39809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.211.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
W20260811 22:31:11.433949 31532 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.211.190:39121,127.18.211.189:39809'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 22:31:11.434896 31532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:11.436074 31532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:11.444909 31538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:11.445389 31544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:11.445405 31539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:11.445595 31532 server_base.cc:1034] running on GCE node
I20260811 22:31:11.446350 31532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:11.447510 31532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:11.448693 31532 hybrid_clock.cc:648] HybridClock initialized: now 1786487471448660 us; error 43 us; skew 500 ppm
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:11.451716 31532 init.cc:377] Logged in from keytab as oryx/127.18.211.189@KRBTEST.COM (short username oryx)
I20260811 22:31:11.452764 31532 webserver.cc:460] Webserver started at http://127.18.211.189:44841/ using document root <none> and password file <none>
I20260811 22:31:11.453487 31532 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:11.453594 31532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:11.453837 31532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:11.456658 31532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/instance:
uuid: "df847b860e2f47a0bccdb34d0b30f30f"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "5ec8a206ca59fd2e38996e1292fae423"
server_key_iv: "44f43936b2cfc5875d045c6e10d82a1f"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.457440 31532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal/instance:
uuid: "df847b860e2f47a0bccdb34d0b30f30f"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "5ec8a206ca59fd2e38996e1292fae423"
server_key_iv: "44f43936b2cfc5875d045c6e10d82a1f"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.463271 31532 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:31:11.466742 31552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.468402 31532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:31:11.468566 31532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
uuid: "df847b860e2f47a0bccdb34d0b30f30f"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "5ec8a206ca59fd2e38996e1292fae423"
server_key_iv: "44f43936b2cfc5875d045c6e10d82a1f"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.468704 31532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:11.483400 31532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:11.487460 31532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:11.487704 31532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:11.498147 31532 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.189:39809
I20260811 22:31:11.498447 31640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.189:39809 every 8 connection(s)
I20260811 22:31:11.499612 31532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
I20260811 22:31:11.502803 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 31532
I20260811 22:31:11.502938 31643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:11.508394 31643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.211.189@KRBTEST.COM
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.189@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:11.518859 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.504571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36220 (local address 127.18.211.189:39809)
0811 22:31:11.505071 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.505096 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.505149 (+ 53us) server_negotiation.cc:407] Connection header received
0811 22:31:11.506462 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.506491 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.507070 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.507480 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.508318 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.509506 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.510521 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.511164 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.511327 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.514151 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.514156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.514166 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.514194 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.516074 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.516520 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.516532 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.516538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.516640 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.516931 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.516936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.516938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.517374 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.517572 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.517880 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.518150 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":320,"spinlock_wait_cycles":2432,"thread_start_us":150,"threads_started":1}
I20260811 22:31:11.519755 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.510206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.18.211.190:39121)
0811 22:31:11.510359 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.510364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.510639 (+ 275us) server_negotiation.cc:407] Connection header received
0811 22:31:11.510771 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.510774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.510824 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.510904 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.511562 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.512264 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.513202 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.513440 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.513556 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.516231 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.516234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.516236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.516260 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.517730 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.518341 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.518345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.518347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.518431 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.518720 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.518723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.518725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.518938 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.519049 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.519551 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.519649 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:31:11.519837 31648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.510123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52468)
0811 22:31:11.510501 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.510548 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.510671 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.510959 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.510964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.511284 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.511434 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.511440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.512406 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.512415 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.513063 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.513073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.513596 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.513601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.513722 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.514264 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.514277 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.516071 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.517854 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.517862 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.517869 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.518239 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.518525 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.518529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.518531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.518621 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.519104 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.519112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.519255 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.519472 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.519642 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260811 22:31:11.521166 31643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.189@KRBTEST.COM for oryx/127.18.211.189@KRBTEST.COM
I20260811 22:31:11.528718 31648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.521669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.0.0.1:36230)
0811 22:31:11.521819 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.521833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.521908 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.522141 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.522144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.522296 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.522370 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.522375 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.523435 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.523438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.523922 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.523929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.524379 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.524383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.524477 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.524928 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.524940 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.526134 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.527514 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.527517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.527518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.527709 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.527937 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.527939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.527940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.528011 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.528326 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.528328 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.528399 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.528476 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.528565 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 22:31:11.528728 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.521755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.18.211.189:39809)
0811 22:31:11.521854 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.521860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.521875 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:31:11.522008 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.522011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.522046 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.522131 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.522479 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.523311 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.524036 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.524274 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.524380 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.526256 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.526259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.526260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.526281 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.527419 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.527791 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.527793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.527794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.527860 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.528097 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.528100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.528101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.528248 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.528320 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.528531 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.528614 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:31:11.531756 31643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Bootstrap starting.
I20260811 22:31:11.534053 31643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:11.534741 31643 log.cc:824] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:11.536620 31643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: No bootstrap required, opened a new log
I20260811 22:31:11.539644 31643 raft_consensus.cc:348] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } }
I20260811 22:31:11.539839 31643 raft_consensus.cc:374] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:11.539881 31643 raft_consensus.cc:729] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df847b860e2f47a0bccdb34d0b30f30f, State: Initialized, Role: FOLLOWER
I20260811 22:31:11.540314 31643 consensus_queue.cc:260] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } }
I20260811 22:31:11.540764 31651 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } } }
I20260811 22:31:11.540901 31651 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:31:11.541585 31643 sys_catalog.cc:564] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:31:11.546605 31677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:31:11.546703 31677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:31:11.548496 31532 master_runner.cc:428] Detected that this master df847b860e2f47a0bccdb34d0b30f30f is joining an existing cluster
I20260811 22:31:11.548558 31532 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.211.189:39809
I20260811 22:31:11.559437 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.551026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.18.211.189:39809)
0811 22:31:11.551130 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.551134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.551478 (+ 344us) server_negotiation.cc:407] Connection header received
0811 22:31:11.551627 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.551630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.551670 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.551763 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.552412 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.553099 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.553998 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.554242 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.554339 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.555578 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.555582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.555585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.555607 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.557391 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.557820 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.557824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.557825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.557905 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.558173 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.558178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.558180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.558776 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.558887 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.559145 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.559304 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":32}
I20260811 22:31:11.559448 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.551049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.0.0.1:36246)
0811 22:31:11.551398 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.551412 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.551529 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.551863 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.551866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.552123 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.552255 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.552263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.553249 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.553253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.553873 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.553882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.554367 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.554371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.554466 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.554974 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.554990 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.555443 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.557499 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.557504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.557506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.557711 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.557992 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.557997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.557999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.558085 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.558886 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.558889 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.558984 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.559087 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.559247 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260811 22:31:11.559706 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.551319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.18.211.190:39121)
0811 22:31:11.551447 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.551453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.551473 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:31:11.551518 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.551521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.551568 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.551645 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.552138 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.552932 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.554227 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.554515 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.554627 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.556011 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.556015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.556018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.556047 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.557800 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.558308 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.558311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.558313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.558406 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.558749 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.558753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.558755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.558978 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.559056 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.559415 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.559537 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:31:11.560144 31692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.550937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52476)
0811 22:31:11.551328 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.551342 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.551434 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.551668 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.551671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.551904 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.552019 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.552025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.553056 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.553060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.554094 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.554106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.554629 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.554633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.555000 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.555481 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.555493 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.555880 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.557936 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.557939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.557941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.558189 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.558520 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.558523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.558525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.558616 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.559135 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.559138 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.559236 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.559840 (+ 604us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.559975 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":24,"thread_start_us":111,"threads_started":1}
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.190@KRBTEST.COM for oryx/127.18.211.189@KRBTEST.COM
I20260811 22:31:11.574894 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.564740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36256 (local address 127.18.211.189:39809)
0811 22:31:11.564885 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.564888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.565614 (+ 726us) server_negotiation.cc:407] Connection header received
0811 22:31:11.565835 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.565838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.565926 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.566024 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.566694 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.567168 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.568052 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.568269 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.568377 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.570895 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.570898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.570900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.570921 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.572669 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.573303 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.573306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.573308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.573401 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.573739 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.573743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.573745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.573961 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.574059 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.574501 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.574731 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:31:11.575156 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.564683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.0.0.1:36256)
0811 22:31:11.565474 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.565515 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.565673 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.566063 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.566069 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.566452 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.566560 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.566565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.567280 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.567285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.567928 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.567937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.568383 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.568387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.568509 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.569052 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.569063 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.570748 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.572795 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.572806 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.572813 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.573156 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.573506 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.573512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.573516 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.573628 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.574078 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.574088 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.574245 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.574407 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.574760 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":104,"threads_started":1}
I20260811 22:31:11.576838 30730 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.211.189:39809: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f"
member_type: NON_VOTER
last_known_addr {
host: "127.18.211.189"
port: 39809
}
attrs {
promote: true
}
}
dest_uuid: "545da267b67449a687813d8a1e5680f4"
cas_config_opid_index: -1
I20260811 22:31:11.577736 30730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } }
I20260811 22:31:11.581756 31701 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER df847b860e2f47a0bccdb34d0b30f30f (127.18.211.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } }
I20260811 22:31:11.582510 31594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.18.211.188@KRBTEST.COM; defaulting to no policy
I20260811 22:31:11.583807 31701 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.211.189:39809
I20260811 22:31:11.584079 31699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } }
I20260811 22:31:11.584154 31594 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Refusing update from remote peer 545da267b67449a687813d8a1e5680f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 22:31:11.584196 31699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:11.584436 31532 master.cc:537] Master@127.18.211.189:39809 shutting down...
I20260811 22:31:11.585561 31699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.188@KRBTEST.COM" created.
W20260811 22:31:11.587795 30706 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 22:31:11.588788 30706 consensus_peers.cc:487] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 -> Peer df847b860e2f47a0bccdb34d0b30f30f (127.18.211.189:39809): Couldn't send request to peer df847b860e2f47a0bccdb34d0b30f30f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:31:11.591607 31532 raft_consensus.cc:2229] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:11.591922 31532 raft_consensus.cc:2258] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:11.592033 31532 tablet_replica.cc:330] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: stopping tablet replica
I20260811 22:31:11.607424 31532 master.cc:552] Master@127.18.211.189:39809 shutdown complete.
I20260811 22:31:11.607502 31532 master_runner.cc:305] Clearing existing system tablet
I20260811 22:31:11.607740 31532 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:11.607816 31532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.188.keytab.
Entry for principal oryx/127.18.211.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.188.keytab.
I20260811 22:31:11.608987 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:45005
--webserver_interface=127.18.211.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.188.keytab
--principal=oryx/127.18.211.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:31:11.609198 31532 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 22:31:11.610272 31708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.610486 31532 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:11.610608 31532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 22:31:11.610641 31532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
uuid: "df847b860e2f47a0bccdb34d0b30f30f"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "5ec8a206ca59fd2e38996e1292fae423"
server_key_iv: "44f43936b2cfc5875d045c6e10d82a1f"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.612949 31532 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:11.615403 31532 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:31:11.615501 31532 log.cc:1196] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 22:31:11.615768 31532 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Deleting consensus metadata
I20260811 22:31:11.615942 31532 master_runner.cc:315] Copying system tablet from 127.18.211.190:39121
I20260811 22:31:11.617297 31532 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: tablet copy: Beginning tablet copy session from remote peer at address 127.18.211.190:39121
I20260811 22:31:11.630169 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.618112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.18.211.190:39121)
0811 22:31:11.618241 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.618246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.621513 (+ 3267us) server_negotiation.cc:407] Connection header received
0811 22:31:11.621653 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.621657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.621723 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.621819 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.622340 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.623210 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.624394 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.624784 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.624883 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.626394 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.626398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.626401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.626432 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.628332 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.628831 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.628835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.628837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.628926 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.629253 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.629256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.629258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.629473 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.629573 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.629871 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.629996 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:31:11.630167 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.617999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52492)
0811 22:31:11.621390 (+ 3391us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.621407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.621544 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.621854 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.621857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.622078 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.622214 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.622222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.623338 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.623342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.624258 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.624269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.624904 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.624907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.625006 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.625749 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.625765 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.626187 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.628455 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.628458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.628460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.628730 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.629022 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.629025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.629027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.629125 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.629575 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.629578 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.629693 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.629804 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.629943 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3310,"thread_start_us":141,"threads_started":1}
I20260811 22:31:11.631323 30770 tablet_copy_service.cc:140] P 545da267b67449a687813d8a1e5680f4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer df847b860e2f47a0bccdb34d0b30f30f ({username='oryx', principal='oryx/127.18.211.189@KRBTEST.COM'} at 127.0.0.1:52492)
I20260811 22:31:11.631516 30770 tablet_copy_service.cc:161] P 545da267b67449a687813d8a1e5680f4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer df847b860e2f47a0bccdb34d0b30f30f at {username='oryx', principal='oryx/127.18.211.189@KRBTEST.COM'} at 127.0.0.1:52492: session id = df847b860e2f47a0bccdb34d0b30f30f-00000000000000000000000000000000
I20260811 22:31:11.633714 30770 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:31:11.635795 31532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:31:11.638422 31532 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: tablet copy: Starting download of 0 data blocks...
I20260811 22:31:11.638710 31532 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: tablet copy: Starting download of 1 WAL segments...
I20260811 22:31:11.641114 31532 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:31:11.643263 30770 tablet_copy_service.cc:342] P 545da267b67449a687813d8a1e5680f4: Request end of tablet copy session df847b860e2f47a0bccdb34d0b30f30f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.211.189@KRBTEST.COM'} at 127.0.0.1:52492
I20260811 22:31:11.643545 30770 tablet_copy_service.cc:433] P 545da267b67449a687813d8a1e5680f4: ending tablet copy session df847b860e2f47a0bccdb34d0b30f30f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer df847b860e2f47a0bccdb34d0b30f30f
I20260811 22:31:11.651758 31532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:11.653651 31721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:11.653674 31729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:11.654161 31532 server_base.cc:1034] running on GCE node
W20260811 22:31:11.654130 31726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:11.654490 31532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:11.654812 31532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:11.655970 31532 hybrid_clock.cc:648] HybridClock initialized: now 1786487471655957 us; error 31 us; skew 500 ppm
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:11.657708 31532 init.cc:377] Logged in from keytab as oryx/127.18.211.189@KRBTEST.COM (short username oryx)
I20260811 22:31:11.658221 31532 webserver.cc:460] Webserver started at http://127.18.211.189:33817/ using document root <none> and password file <none>
I20260811 22:31:11.658439 31532 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:11.658495 31532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:11.659941 31532 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:11.661036 31743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.661363 31532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:11.661463 31532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
uuid: "df847b860e2f47a0bccdb34d0b30f30f"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "5ec8a206ca59fd2e38996e1292fae423"
server_key_iv: "44f43936b2cfc5875d045c6e10d82a1f"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.661551 31532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:11.703967 31532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:11.706565 31532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:11.711606 31532 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.189:39809
I20260811 22:31:11.711642 31848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.189:39809 every 8 connection(s)
I20260811 22:31:11.712292 31532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
I20260811 22:31:11.715014 31853 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:31:11.716174 31853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Bootstrap starting.
I20260811 22:31:11.727650 31853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:31:11.727996 31853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: Bootstrap complete.
I20260811 22:31:11.728645 31853 raft_consensus.cc:348] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } }
I20260811 22:31:11.728798 31853 raft_consensus.cc:729] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: df847b860e2f47a0bccdb34d0b30f30f, State: Initialized, Role: LEARNER
I20260811 22:31:11.728928 31853 consensus_queue.cc:260] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } }
I20260811 22:31:11.729362 31858 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } }
I20260811 22:31:11.729481 31853 sys_catalog.cc:564] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:11.729490 31858 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: LEARNER
I20260811 22:31:11.732645 31882 catalog_manager.cc:1259] Loaded cluster ID: 72d2d6dc099a40d1bc2fab835e24ead2
I20260811 22:31:11.732960 31882 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: loading cluster ID for follower catalog manager: success
I20260811 22:31:11.735204 31882 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: acquiring CA information for follower catalog manager: success
I20260811 22:31:11.736577 31882 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:31:11.753096 31713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:11.753448 31713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:11.753504 31713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:11.757900 31713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:11.757964 31713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:11.757992 31713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:11.758009 31713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:11.758025 31713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:11.764109 31713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.188.keytab
--principal=oryx/127.18.211.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:45005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.211.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:11.765296 31713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:11.766446 31713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:11.775359 31889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:11.775383 31890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:11.775514 31713 server_base.cc:1034] running on GCE node
W20260811 22:31:11.775732 31893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:11.776342 31713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:11.777532 31713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:11.778697 31713 hybrid_clock.cc:648] HybridClock initialized: now 1786487471778675 us; error 34 us; skew 500 ppm
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:11.781757 31713 init.cc:377] Logged in from keytab as oryx/127.18.211.188@KRBTEST.COM (short username oryx)
I20260811 22:31:11.782688 31713 webserver.cc:460] Webserver started at http://127.18.211.188:40639/ using document root <none> and password file <none>
I20260811 22:31:11.783260 31713 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:11.783351 31713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:11.783591 31713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:11.785714 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/instance:
uuid: "744bd6308ad54d5286826c5b4f01ca34"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "bbab55ff70f247507a96f3f4c0e0dc63"
server_key_iv: "6080a1f1fd7b83f4e8dc00f69178dd5a"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.785997 31067 heartbeater.cc:503] Master 127.18.211.190:39121 was elected leader, sending a full tablet report...
I20260811 22:31:11.786365 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal/instance:
uuid: "744bd6308ad54d5286826c5b4f01ca34"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "bbab55ff70f247507a96f3f4c0e0dc63"
server_key_iv: "6080a1f1fd7b83f4e8dc00f69178dd5a"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.790616 31713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:31:11.793128 31905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.794451 31713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:11.794600 31713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
uuid: "744bd6308ad54d5286826c5b4f01ca34"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "bbab55ff70f247507a96f3f4c0e0dc63"
server_key_iv: "6080a1f1fd7b83f4e8dc00f69178dd5a"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.794723 31713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:11.816847 31713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:11.820449 31713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:11.820704 31713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:11.828397 31713 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.188:45005
I20260811 22:31:11.828737 32005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.188:45005 every 8 connection(s)
I20260811 22:31:11.829591 31713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
I20260811 22:31:11.832816 32006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:11.836903 32006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:31:11.838093 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 31713
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.188@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.211.188@KRBTEST.COM
I20260811 22:31:11.851517 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.839110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52508 (local address 127.18.211.190:39121)
0811 22:31:11.839264 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.839269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.840617 (+ 1348us) server_negotiation.cc:407] Connection header received
0811 22:31:11.841034 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.841037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.841103 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.841207 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.842192 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.843062 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.844111 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.844335 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.844451 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.847274 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.847278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.847280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.847307 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.849083 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.849743 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.849748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.849752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.849849 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.850273 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.850276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.850277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.850518 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.850626 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.851198 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.851334 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:31:11.852407 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.839475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.18.211.188:45005)
0811 22:31:11.839951 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.839966 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.840000 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:31:11.840747 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.840765 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.840969 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.841183 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.842058 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.843364 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.844221 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.844650 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.844770 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.847753 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.847760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.847776 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.847805 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.849727 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.850240 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.850248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.850253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.850347 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.850616 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.850620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.850621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.850888 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.851051 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.851311 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.851672 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":201,"server-negotiator.queue_time_us":344,"thread_start_us":177,"threads_started":1}
I20260811 22:31:11.852411 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.839475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52508)
0811 22:31:11.840470 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.840509 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.840829 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.841414 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.841420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.841781 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.842038 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.842051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.843197 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.843200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.843964 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.843973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.844500 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.844504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.844732 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.845397 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.845423 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.847101 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.849199 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.849206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.849256 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.849639 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.849964 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.849966 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.849968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.850155 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.850626 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.850634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.850900 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.851089 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.851458 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":53,"spinlock_wait_cycles":49920,"thread_start_us":177,"threads_started":1}
I20260811 22:31:11.853708 32006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.188@KRBTEST.COM for oryx/127.18.211.189@KRBTEST.COM
I20260811 22:31:11.862146 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.854275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.0.0.1:36262)
0811 22:31:11.854465 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.854482 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.854549 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.854947 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.854949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.855079 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.855198 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.855203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.856027 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.856030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.856579 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.856586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.856990 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.856993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.857064 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.857601 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.857612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.858872 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.860677 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.860680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.860681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.860861 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.861126 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.861128 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.861129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.861207 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.861615 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.861617 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.861747 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.861831 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.861995 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:31:11.862160 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.854398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.18.211.189:39809)
0811 22:31:11.854730 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.854734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.854747 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:11.854786 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.854788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.854841 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.854915 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.855301 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.855938 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.856681 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.856895 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.856973 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.859014 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.859017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.859019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.859043 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.860578 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.860967 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.860970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.860971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.861046 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.861348 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.861351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.861352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.861532 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.861609 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.861933 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.862038 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260811 22:31:11.863003 32006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487471, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.188@KRBTEST.COM for oryx/127.18.211.188@KRBTEST.COM
I20260811 22:31:11.870306 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.863469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.0.0.1:49778)
0811 22:31:11.863578 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.863590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.863711 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.863898 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.863900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.864026 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.864097 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.864102 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.864775 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.864778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.865344 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.865353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.865761 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.865763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.865822 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.866327 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.866339 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.867516 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.868978 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.868980 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.868981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.869165 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.869500 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.869502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.869503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.869580 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.869904 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.869906 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.869981 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.870059 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.870156 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:31:11.870335 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.863585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.18.211.188:45005)
0811 22:31:11.863705 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.863709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.863720 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:31:11.863771 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.863773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.863806 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.863869 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.864200 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.864685 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.865443 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.865664 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.865746 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.867643 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.867646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.867648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.867669 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.868876 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.869341 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.869344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.869345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.869417 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.869657 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.869659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.869660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.869825 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.869898 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.870117 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.870213 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:31:11.873893 32006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Bootstrap starting.
I20260811 22:31:11.876320 32006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:11.877139 32006 log.cc:824] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:11.879127 32006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: No bootstrap required, opened a new log
I20260811 22:31:11.882524 32006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } }
I20260811 22:31:11.882773 32006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:11.882858 32006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744bd6308ad54d5286826c5b4f01ca34, State: Initialized, Role: FOLLOWER
I20260811 22:31:11.883315 32006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } }
I20260811 22:31:11.883823 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } } }
I20260811 22:31:11.883942 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:31:11.884584 32006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:31:11.889604 32036 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:31:11.889716 32036 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:31:11.893152 31713 master_runner.cc:428] Detected that this master 744bd6308ad54d5286826c5b4f01ca34 is joining an existing cluster
I20260811 22:31:11.893252 31713 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.211.188:45005
I20260811 22:31:11.905916 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 30838
I20260811 22:31:11.920032 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.895899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.18.211.188:45005)
0811 22:31:11.896002 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.896006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.896477 (+ 471us) server_negotiation.cc:407] Connection header received
0811 22:31:11.896630 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.896634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.896669 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.896730 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.897466 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.898257 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.899537 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.899927 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.900027 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.901344 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.901351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.901357 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.901389 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.917059 (+ 15670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.917617 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.917621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.917623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.917979 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.918732 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.918740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.918742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.919204 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.919295 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.919695 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.919840 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":40}
I20260811 22:31:11.920574 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.896077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.0.0.1:49792)
0811 22:31:11.896397 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.896411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.896536 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.896871 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.896874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.897153 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.897346 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.897355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.898403 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.898406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.899362 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.899373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.900054 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.900058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.900156 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.900704 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.900719 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.901197 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.917200 (+ 16003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.917204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.917206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.917505 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.918089 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.918092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.918094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.918402 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.919342 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.919345 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.919441 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.920268 (+ 827us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.920408 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":71,"thread_start_us":86,"threads_started":1}
I20260811 22:31:11.923079 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.895573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.18.211.189:39809)
0811 22:31:11.895704 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.895711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.896368 (+ 657us) server_negotiation.cc:407] Connection header received
0811 22:31:11.896542 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.896548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.896604 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.896659 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.897455 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.898204 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.900678 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.900906 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.900965 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.917832 (+ 16867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.917839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.917845 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.917884 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.920652 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.921700 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.921704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.921706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.921798 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.922107 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.922110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.922112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.922342 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.922427 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.922786 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.922923 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:31:11.923555 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.895484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.0.0.1:36270)
0811 22:31:11.896290 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.896303 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.896431 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.896810 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.896813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.897093 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.897277 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.897286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.898334 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.898337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.900241 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.900251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.901093 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.901095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.901160 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.901660 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.901670 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.916130 (+ 14460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.921313 (+ 5183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.921316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.921319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.921561 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.921918 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.921921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.921923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.922005 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.922487 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.922490 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.922579 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.923235 (+ 656us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.923382 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":43,"thread_start_us":92,"threads_started":1}
I20260811 22:31:11.924095 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.895859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.18.211.190:39121)
0811 22:31:11.895984 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.895988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.896002 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:11.896159 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.896164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.896222 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.896297 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.897097 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.898073 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.899729 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.900054 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.900127 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.917306 (+ 17179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.917313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.917319 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.917362 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.919719 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.922462 (+ 2743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.922469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.922474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.922575 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.922924 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.922930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.922934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.923276 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.923360 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.923774 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.923910 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:31:11.924770 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.895374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52516)
0811 22:31:11.895889 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.895905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.896041 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.896434 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.896437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.896769 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.896950 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.896960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.898248 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.898254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.899583 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.899594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.900274 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.900280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.900414 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.914989 (+ 14575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.915016 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.915635 (+ 619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.920838 (+ 5203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.920844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.920849 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.922333 (+ 1484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.922707 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.922712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.922717 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.922813 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.923432 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.923435 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.923546 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.924334 (+ 788us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.924491 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":43,"thread_start_us":113,"threads_started":1}
I20260811 22:31:11.926486 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:38915
--local_ip_for_outbound_sockets=127.18.211.129
--tserver_master_addrs=127.18.211.190:39121
--webserver_port=36799
--webserver_interface=127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab
--principal=oryx/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:31:11 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487470, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.190@KRBTEST.COM for oryx/127.18.211.188@KRBTEST.COM
I20260811 22:31:11.939944 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.928736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.0.0.1:49794)
0811 22:31:11.928972 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.928986 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.929060 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.929798 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.929801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.929996 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.930110 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.930118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.931269 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.931272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.932200 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.932212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.932891 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.932895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.932996 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.933558 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.933573 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.935334 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.937746 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.937751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.937756 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.938046 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.938407 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.938413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.938418 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.938617 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.939219 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.939225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.939354 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.939479 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.939668 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260811 22:31:11.940567 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.929186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.18.211.188:45005)
0811 22:31:11.929478 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.929484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.929499 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:31:11.929618 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.929623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.929683 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.929763 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.930255 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.931125 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.932335 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.932758 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.932892 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.935491 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.935498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.935502 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.935532 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.937563 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.938180 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.938184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.938186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.938284 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.938773 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.938777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.938781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.939056 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.939165 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.940204 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.940349 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260811 22:31:11.941942 30730 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.211.188:45005: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34"
member_type: NON_VOTER
last_known_addr {
host: "127.18.211.188"
port: 45005
}
attrs {
promote: true
}
}
dest_uuid: "545da267b67449a687813d8a1e5680f4"
cas_config_opid_index: 5
I20260811 22:31:11.942623 30730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } }
I20260811 22:31:11.943786 31699 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.945437 31699 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.211.188:45005
I20260811 22:31:11.945637 31699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.945750 31699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:11.946206 31713 master.cc:537] Master@127.18.211.188:45005 shutting down...
I20260811 22:31:11.946475 31950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:11.948058 31950 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Refusing update from remote peer 545da267b67449a687813d8a1e5680f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:31:11.948833 31701 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:31:11.953388 30706 consensus_peers.cc:487] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 -> Peer 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188:45005): Couldn't send request to peer 744bd6308ad54d5286826c5b4f01ca34. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:31:11.954679 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.945684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.18.211.189:39809)
0811 22:31:11.945821 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.945827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.949461 (+ 3634us) server_negotiation.cc:407] Connection header received
0811 22:31:11.949671 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.949674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.949751 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.949821 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:11.950213 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.951083 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.952903 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.953625 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.953730 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.953902 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.954447 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.954543 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:31:11.955058 31713 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:11.955336 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.945604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.0.0.1:36282)
0811 22:31:11.949367 (+ 3763us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.949384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.949546 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.949983 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.949985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.950002 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:11.950093 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.950099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.951242 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.951246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.952749 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.952763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.954153 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.954159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.954267 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.954318 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.955070 (+ 752us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.955204 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3668}
I20260811 22:31:11.955476 31713 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:11.955606 31713 tablet_replica.cc:330] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: stopping tablet replica
I20260811 22:31:11.956084 31795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 LEARNER]: Refusing update from remote peer 545da267b67449a687813d8a1e5680f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:31:11.956576 31701 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 22:31:11.958840 31795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.960976 32059 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.961093 32059 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: LEARNER
I20260811 22:31:11.961406 31858 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545da267b67449a687813d8a1e5680f4. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: NON_VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: true } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.961527 31858 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: LEARNER
I20260811 22:31:11.962466 31701 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: attempting to promote NON_VOTER df847b860e2f47a0bccdb34d0b30f30f to VOTER
I20260811 22:31:11.962977 31701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } }
I20260811 22:31:11.965274 31795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 LEARNER]: Refusing update from remote peer 545da267b67449a687813d8a1e5680f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:31:11.965665 31701 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 22:31:11.966461 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.965451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.0.0.1:49804)
0811 22:31:11.966295 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.966375 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:45005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":763}
W20260811 22:31:11.966550 30706 consensus_peers.cc:487] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 -> Peer 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188:45005): Couldn't send request to peer 744bd6308ad54d5286826c5b4f01ca34. Status: Network error: Client connection negotiation failed: client connection to 127.18.211.188:45005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:31:11.967729 31701 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, df847b860e2f47a0bccdb34d0b30f30f (127.18.211.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.969406 31701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.969520 31701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:11.969851 31701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.969947 31701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:11.970614 31795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, df847b860e2f47a0bccdb34d0b30f30f (127.18.211.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.972204 31858 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:11.972322 31858 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:31:11.975874 31713 master.cc:552] Master@127.18.211.188:45005 shutdown complete.
I20260811 22:31:11.975936 31713 master_runner.cc:305] Clearing existing system tablet
I20260811 22:31:11.976207 31713 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:11.976272 31713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:11.977671 31713 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 22:31:11.978682 32064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:11.978899 31713 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:11.979190 31713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:11.979277 31713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
uuid: "744bd6308ad54d5286826c5b4f01ca34"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "bbab55ff70f247507a96f3f4c0e0dc63"
server_key_iv: "6080a1f1fd7b83f4e8dc00f69178dd5a"
server_key_version: "encryptionkey@0"
I20260811 22:31:11.981732 31713 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:11.984395 31713 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:31:11.984565 31713 log.cc:1196] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 22:31:11.984894 31713 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Deleting consensus metadata
I20260811 22:31:11.985122 31713 master_runner.cc:315] Copying system tablet from 127.18.211.190:39121
I20260811 22:31:11.986364 31713 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: tablet copy: Beginning tablet copy session from remote peer at address 127.18.211.190:39121
I20260811 22:31:11.996779 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.987276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52522 (local address 127.18.211.190:39121)
0811 22:31:11.987403 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:11.987408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:11.987428 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:31:11.987477 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:11.987480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:11.987529 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:11.987630 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:11.988252 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.989092 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.990409 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.990762 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.990868 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.992780 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:11.992783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:11.992787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:11.992816 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:11.994753 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.995433 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.995437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.995439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.995521 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:11.995834 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:11.995837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:11.995839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:11.996046 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:11.996114 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:11.996481 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 22:31:11.996619 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:31:11.997356 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:11.986963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52522)
0811 22:31:11.987249 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:11.987266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:11.987383 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:11.987657 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:11.987660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:11.987885 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:11.988113 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.988126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.989261 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.989267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.990258 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:11.990273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:11.990908 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:11.990914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:11.991061 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:11.992172 (+ 1111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:11.992187 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:11.992622 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:11.994875 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.994878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.994880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.995129 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.995650 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:11.995653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:11.995655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:11.995738 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:11.996186 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:11.996188 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:11.996284 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 22:31:11.996983 (+ 699us) client_negotiation.cc:240] Negotiation successful
0811 22:31:11.997139 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
I20260811 22:31:11.997846 30770 tablet_copy_service.cc:140] P 545da267b67449a687813d8a1e5680f4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 744bd6308ad54d5286826c5b4f01ca34 ({username='oryx', principal='oryx/127.18.211.188@KRBTEST.COM'} at 127.0.0.1:52522)
I20260811 22:31:11.997965 30770 tablet_copy_service.cc:161] P 545da267b67449a687813d8a1e5680f4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 744bd6308ad54d5286826c5b4f01ca34 at {username='oryx', principal='oryx/127.18.211.188@KRBTEST.COM'} at 127.0.0.1:52522: session id = 744bd6308ad54d5286826c5b4f01ca34-00000000000000000000000000000000
I20260811 22:31:11.999061 30770 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:31:12.000362 31713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:31:12.002713 31713 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: tablet copy: Starting download of 0 data blocks...
I20260811 22:31:12.002923 31713 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: tablet copy: Starting download of 1 WAL segments...
I20260811 22:31:12.004560 31713 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:31:12.006428 30770 tablet_copy_service.cc:342] P 545da267b67449a687813d8a1e5680f4: Request end of tablet copy session 744bd6308ad54d5286826c5b4f01ca34-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.211.188@KRBTEST.COM'} at 127.0.0.1:52522
I20260811 22:31:12.006503 30770 tablet_copy_service.cc:433] P 545da267b67449a687813d8a1e5680f4: ending tablet copy session 744bd6308ad54d5286826c5b4f01ca34-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 744bd6308ad54d5286826c5b4f01ca34
I20260811 22:31:12.013892 31713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:12.015765 32074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:12.017623 31283 heartbeater.cc:503] Master 127.18.211.190:39121 was elected leader, sending a full tablet report...
I20260811 22:31:12.019094 31713 server_base.cc:1034] running on GCE node
W20260811 22:31:12.019764 32086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:12.021384 32078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:12.021751 31713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:12.022104 31713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:12.023260 31713 hybrid_clock.cc:648] HybridClock initialized: now 1786487472023232 us; error 44 us; skew 500 ppm
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:12.025031 31713 init.cc:377] Logged in from keytab as oryx/127.18.211.188@KRBTEST.COM (short username oryx)
I20260811 22:31:12.025594 31713 webserver.cc:460] Webserver started at http://127.18.211.188:46175/ using document root <none> and password file <none>
I20260811 22:31:12.025799 31713 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:12.025848 31713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:12.027374 31713 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:12.028519 32101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.028893 31713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:12.028972 31713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
uuid: "744bd6308ad54d5286826c5b4f01ca34"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "bbab55ff70f247507a96f3f4c0e0dc63"
server_key_iv: "6080a1f1fd7b83f4e8dc00f69178dd5a"
server_key_version: "encryptionkey@0"
I20260811 22:31:12.029060 31713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:12.040417 31713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:12.042991 31713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:12.048262 31713 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.188:45005
I20260811 22:31:12.048286 32192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.188:45005 every 8 connection(s)
I20260811 22:31:12.048935 31713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
I20260811 22:31:12.050760 32193 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:31:12.051925 32193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Bootstrap starting.
I20260811 22:31:12.064651 32193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:31:12.065062 32193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Bootstrap complete.
I20260811 22:31:12.065719 32193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } }
I20260811 22:31:12.065881 32193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 744bd6308ad54d5286826c5b4f01ca34, State: Initialized, Role: LEARNER
I20260811 22:31:12.066066 32193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } }
I20260811 22:31:12.066460 32201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:12.066606 32193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:12.066648 32201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:31:12.070770 32221 catalog_manager.cc:1259] Loaded cluster ID: 72d2d6dc099a40d1bc2fab835e24ead2
I20260811 22:31:12.070835 32221 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: loading cluster ID for follower catalog manager: success
I20260811 22:31:12.073137 32221 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: acquiring CA information for follower catalog manager: success
I20260811 22:31:12.074824 32221 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:31:12.083477 32057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:12.083797 32057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:12.083909 32057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:12.088027 32057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:12.088115 32057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:12.088214 32057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:12.093524 32057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab
--principal=oryx/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:38915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=36799
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:12.094611 32057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:12.095628 32057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:12.105434 32228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:12.105437 32226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:12.105500 32233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:12.105681 32057 server_base.cc:1034] running on GCE node
I20260811 22:31:12.106135 32057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:12.106773 32057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:12.107930 32057 hybrid_clock.cc:648] HybridClock initialized: now 1786487472107898 us; error 42 us; skew 500 ppm
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:12.110788 32057 init.cc:377] Logged in from keytab as oryx/127.18.211.129@KRBTEST.COM (short username oryx)
I20260811 22:31:12.111613 32057 webserver.cc:460] Webserver started at http://127.18.211.129:36799/ using document root <none> and password file <none>
I20260811 22:31:12.112167 32057 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:12.112243 32057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:12.116147 32057 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:12.118202 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.119489 32057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:12.119656 32057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "e3cdc0758af94b74b610e3093a7b8a5a"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "049f542f7879281404a2b22244fa9d6b"
server_key_iv: "f91e2e3c427a609d46f31ca5314c9619"
server_key_version: "encryptionkey@0"
I20260811 22:31:12.120263 32057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:12.134321 32057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:12.137455 32057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:12.137665 32057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:12.138382 32057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:12.139264 32057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:12.139336 32057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.139410 32057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:12.139461 32057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.149438 32057 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:38915
I20260811 22:31:12.149466 32435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:38915 every 8 connection(s)
I20260811 22:31:12.150477 32057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:12.156618 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 32057
I20260811 22:31:12.156811 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 31072
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.189@KRBTEST.COM
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.188@KRBTEST.COM
I20260811 22:31:12.170414 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.152406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:40387 (local address 127.18.211.188:45005)
0811 22:31:12.152679 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.152684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.153814 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 22:31:12.154628 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.154632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.154691 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.154761 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.156366 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.157450 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.159312 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.159880 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.160045 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.163938 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.163942 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.163944 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.163980 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.167764 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.168437 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.168443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.168447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.168538 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.168948 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.168953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.168957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.169244 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.169316 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.170010 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.170206 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260811 22:31:12.172897 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.152826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.18.211.129:40387)
0811 22:31:12.153715 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.153731 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.154419 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.154993 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.155003 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.155578 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.156192 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.156209 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.157582 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.157586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.159165 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.159177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.160036 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.160040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.160130 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.162339 (+ 2209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.162359 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.163790 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.167915 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.167925 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.167948 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.168327 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.168652 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.168657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.168661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.168849 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.169399 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.169408 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.169750 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.170840 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.171227 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":314,"spinlock_wait_cycles":27008,"thread_start_us":163,"threads_started":1}
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:12.174497 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.154058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:51129 (local address 127.18.211.189:39809)
0811 22:31:12.154194 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.154200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.154254 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:31:12.154628 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.154632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.154708 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.154765 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.156366 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.157185 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.159316 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.159583 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.159705 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.163052 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.163056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.163059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.163090 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.166851 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.170611 (+ 3760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.170615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.170617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.170719 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.172262 (+ 1543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.172269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.172273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.172945 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.173059 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.174102 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.174262 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:31:12.175113 32438 heartbeater.cc:347] Connected to a master server at 127.18.211.188:45005
I20260811 22:31:12.175316 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.153924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.18.211.129:51129)
0811 22:31:12.154177 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.154192 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.154418 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.154992 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.155003 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.155564 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.156192 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.156209 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.157390 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.157393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.159165 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.159177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.159758 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.159763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.159984 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.160909 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.160934 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.162838 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.170165 (+ 7327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.170168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.170170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.170403 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.172037 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.172040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.172042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.172137 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.173761 (+ 1624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.173767 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.173891 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.174836 (+ 945us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.175014 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":221,"spinlock_wait_cycles":37760,"thread_start_us":90,"threads_started":1}
I20260811 22:31:12.175398 32438 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.175940 32438 heartbeater.cc:511] Master 127.18.211.188:45005 requested a full tablet report, sending...
I20260811 22:31:12.177744 32129 ts_manager.cc:194] Registered new tserver with Master: e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:12.178422 32439 heartbeater.cc:347] Connected to a master server at 127.18.211.189:39809
I20260811 22:31:12.178491 32439 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.178648 32439 heartbeater.cc:511] Master 127.18.211.189:39809 requested a full tablet report, sending...
I20260811 22:31:12.178722 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.152464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:43335 (local address 127.18.211.190:39121)
0811 22:31:12.152597 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.152601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.153529 (+ 928us) server_negotiation.cc:407] Connection header received
0811 22:31:12.154626 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.154629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.154692 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.154795 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.156343 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.157172 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.171562 (+ 14390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.171969 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.172065 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.174634 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.174638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.174641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.174676 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.176630 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.177131 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.177138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.177142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.177267 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.177560 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.177566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.177570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.177861 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.177962 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.178327 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.178468 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:31:12.179342 32446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.152810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.129:43335)
0811 22:31:12.153348 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.153401 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.154399 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.154991 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.155001 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.155564 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.156191 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.156207 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.170537 (+ 14330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.170540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.171395 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.171406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.172271 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.172275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.172368 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.172930 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.172945 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.174460 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.176745 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.176751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.176756 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.177002 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.177355 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.177358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.177360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.177453 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.178013 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.178019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.178128 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.178959 (+ 831us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.179135 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":78,"spinlock_wait_cycles":1920,"thread_start_us":643,"threads_started":1}
I20260811 22:31:12.180049 32437 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39121
I20260811 22:31:12.180143 32437 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.180174 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:35859
--local_ip_for_outbound_sockets=127.18.211.130
--tserver_master_addrs=127.18.211.190:39121
--webserver_port=40869
--webserver_interface=127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab
--principal=oryx/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:31:12.180213 31780 ts_manager.cc:194] Registered new tserver with Master: e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:12.180305 32437 heartbeater.cc:511] Master 127.18.211.190:39121 requested a full tablet report, sending...
I20260811 22:31:12.181200 30730 ts_manager.cc:194] Re-registered known tserver with Master: e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:12.182114 30730 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:43335
I20260811 22:31:12.265619 31524 heartbeater.cc:503] Master 127.18.211.190:39121 was elected leader, sending a full tablet report...
W20260811 22:31:12.298672 32449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:12.298980 32449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:12.299057 32449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:12.303177 32449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:12.303269 32449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:12.303357 32449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:12.308730 32449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab
--principal=oryx/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:35859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=40869
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:12.309885 32449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:12.310858 32449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:12.320928 32458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:12.320932 32457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:12.320974 32461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:12.321194 32449 server_base.cc:1034] running on GCE node
I20260811 22:31:12.321789 32449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:12.322446 32449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:12.323613 32449 hybrid_clock.cc:648] HybridClock initialized: now 1786487472323591 us; error 32 us; skew 500 ppm
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:12.326613 32449 init.cc:377] Logged in from keytab as oryx/127.18.211.130@KRBTEST.COM (short username oryx)
I20260811 22:31:12.327584 32449 webserver.cc:460] Webserver started at http://127.18.211.130:40869/ using document root <none> and password file <none>
I20260811 22:31:12.328169 32449 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:12.328271 32449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:12.332486 32449 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:12.334648 32476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.335913 32449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:31:12.336053 32449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "93a01bbb5e8f4acdbdd519dcc69ca548"
format_stamp: "Formatted at 2026-08-11 22:31:10 on dist-test-slave-8hhm"
server_key: "d8d890a615eeb38ac8ec6e546e589d5c"
server_key_iv: "52b9cb29955aba8db16e4d2d3821029c"
server_key_version: "encryptionkey@0"
I20260811 22:31:12.336615 32449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:12.347926 32449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:12.351513 32449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:12.351751 32449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:12.352381 32449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:12.353364 32449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:12.353433 32449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.353519 32449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:12.353574 32449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.364545 32449 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:35859
I20260811 22:31:12.364564 32668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:35859 every 8 connection(s)
I20260811 22:31:12.365758 32449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:12.368207 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 32449
I20260811 22:31:12.368386 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 31290
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.188@KRBTEST.COM
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.189@KRBTEST.COM
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:12.389315 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.367714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:57437 (local address 127.18.211.190:39121)
0811 22:31:12.367854 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.367859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.374195 (+ 6336us) server_negotiation.cc:407] Connection header received
0811 22:31:12.374436 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.374442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.374516 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.374586 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.376494 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.377578 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.379253 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.379737 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.379851 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.384525 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.384532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.384537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.384577 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.386454 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.387155 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.387161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.387166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.387285 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.387757 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.387761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.387762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.388023 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.388101 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.388933 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.389092 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:31:12.390596 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.371823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:53655 (local address 127.18.211.188:45005)
0811 22:31:12.371955 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.371961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.375081 (+ 3120us) server_negotiation.cc:407] Connection header received
0811 22:31:12.375245 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.375250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.375309 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.375367 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.376069 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.376757 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.378751 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.379098 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.379200 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.383739 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.383744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.383747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.383779 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.388104 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.388922 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.388926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.388928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.389028 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.389388 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.389392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.389393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.389629 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.389726 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.390310 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.390454 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:31:12.390906 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.367718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:49597 (local address 127.18.211.189:39809)
0811 22:31:12.367854 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.367860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.372066 (+ 4206us) server_negotiation.cc:407] Connection header received
0811 22:31:12.373832 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.373839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.373907 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.373967 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.376454 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.377185 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.378758 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.379098 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.379195 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.384305 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.384311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.384314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.384344 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.387135 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.388891 (+ 1756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.388898 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.388903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.389037 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.389528 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.389535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.389540 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.389877 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.389972 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.390477 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.390651 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:12.391233 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.368235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.130:57437)
0811 22:31:12.374065 (+ 5830us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.374085 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.374289 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.374784 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.374790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.375127 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.375919 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.375934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.377758 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.377764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.379046 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.379062 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.380088 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.380096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.380282 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.380742 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.380766 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.384355 (+ 3589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.386604 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.386613 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.386635 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.387025 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.387415 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.387418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.387420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.387630 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.388207 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.388213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.388601 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.389591 (+ 990us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.390011 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5712,"mutex_wait_us":39,"spinlock_wait_cycles":896,"thread_start_us":133,"threads_started":1}
I20260811 22:31:12.391732 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.372043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.18.211.130:53655)
0811 22:31:12.374997 (+ 2954us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.375012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.375144 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.375489 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.375491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.375630 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.375913 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.375927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.376925 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.376928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.378597 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.378608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.379424 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.379428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.379784 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.380887 (+ 1103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.380904 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.383540 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.388217 (+ 4677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.388220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.388222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.388450 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.389149 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.389152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.389154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.389271 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.390028 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.390031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.390119 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.391126 (+ 1007us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.391287 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2880,"mutex_wait_us":259,"spinlock_wait_cycles":1408,"thread_start_us":1715,"threads_started":1}
I20260811 22:31:12.391819 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.368236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.18.211.130:49597)
0811 22:31:12.371851 (+ 3615us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.371936 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.373502 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.374385 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.374396 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.376116 (+ 1720us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.376298 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.376311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.377365 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.377368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.378597 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.378608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.379424 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.379428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.379782 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.380622 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.380646 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.383984 (+ 3338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.388541 (+ 4557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.388544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.388546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.388775 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.389237 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.389244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.389249 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.389380 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.390065 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.390071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.390221 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.391260 (+ 1039us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.391463 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3298,"mutex_wait_us":166,"spinlock_wait_cycles":4480,"thread_start_us":3020,"threads_started":1}
I20260811 22:31:12.392750 32669 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39121
I20260811 22:31:12.393021 32669 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.393046 32671 heartbeater.cc:347] Connected to a master server at 127.18.211.189:39809
I20260811 22:31:12.393105 32671 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.393249 32670 heartbeater.cc:347] Connected to a master server at 127.18.211.188:45005
I20260811 22:31:12.393349 32670 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.393718 32669 heartbeater.cc:511] Master 127.18.211.190:39121 requested a full tablet report, sending...
I20260811 22:31:12.393723 32670 heartbeater.cc:511] Master 127.18.211.188:45005 requested a full tablet report, sending...
I20260811 22:31:12.394338 32671 heartbeater.cc:511] Master 127.18.211.189:39809 requested a full tablet report, sending...
I20260811 22:31:12.395066 31780 ts_manager.cc:194] Registered new tserver with Master: 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130:35859)
I20260811 22:31:12.395085 30730 ts_manager.cc:194] Re-registered known tserver with Master: 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130:35859)
I20260811 22:31:12.396137 30730 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:57437
I20260811 22:31:12.396973 32129 ts_manager.cc:194] Registered new tserver with Master: 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130:35859)
I20260811 22:31:12.408712 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:42535
--local_ip_for_outbound_sockets=127.18.211.131
--tserver_master_addrs=127.18.211.190:39121
--webserver_port=41503
--webserver_interface=127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab
--principal=oryx/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:31:12.433470 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.429342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.0.0.1:49806)
0811 22:31:12.429496 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.429514 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.429632 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.429977 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.429980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.430000 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:12.430108 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.430114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.431219 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.431222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.432133 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.432143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.433026 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.433028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.433126 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.433162 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.433283 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.433349 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 22:31:12.433491 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.429435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.18.211.188:45005)
0811 22:31:12.429572 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.429576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.429593 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:31:12.429784 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.429787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.429855 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.429963 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:12.430288 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.431090 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.432277 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.432896 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.433030 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.433203 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.433293 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.433365 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:31:12.434253 32150 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 22:31:12.434800 32201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545da267b67449a687813d8a1e5680f4. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:12.434952 32201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: This master's current role is: LEARNER
W20260811 22:31:12.530920 32679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:12.531183 32679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:12.531262 32679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:12.535663 32679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:12.535734 32679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:12.535830 32679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:12.541493 32679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab
--principal=oryx/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:42535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=41503
--tserver_master_addrs=127.18.211.190:39121,127.18.211.189:39809,127.18.211.188:45005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:12.542598 32679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:12.543682 32679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:12.554028 32688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:12.553977 32687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:12.553974 32696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:12.554769 32679 server_base.cc:1034] running on GCE node
I20260811 22:31:12.555156 32679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:12.555855 32679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:12.557034 32679 hybrid_clock.cc:648] HybridClock initialized: now 1786487472556993 us; error 49 us; skew 500 ppm
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:12.559932 32679 init.cc:377] Logged in from keytab as oryx/127.18.211.131@KRBTEST.COM (short username oryx)
I20260811 22:31:12.560931 32679 webserver.cc:460] Webserver started at http://127.18.211.131:41503/ using document root <none> and password file <none>
I20260811 22:31:12.561733 32679 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:12.561854 32679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:12.566278 32679 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:12.568638 32707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.570024 32679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:12.570130 32679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "c4361fe589634297aeca20acc858faf0"
format_stamp: "Formatted at 2026-08-11 22:31:11 on dist-test-slave-8hhm"
server_key: "86c4a9dffcf7fdd66ee34d170e95012a"
server_key_iv: "3d968fea3e0e8c6857626398dbfe477d"
server_key_version: "encryptionkey@0"
I20260811 22:31:12.570683 32679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:12.591923 32679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:12.594955 32679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:12.595134 32679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:12.595729 32679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:12.596719 32679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:12.596774 32679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.596820 32679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:12.596843 32679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:12.606634 32679 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:42535
I20260811 22:31:12.606657 422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:42535 every 8 connection(s)
I20260811 22:31:12.607721 32679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:12.616547 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 32679
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.189@KRBTEST.COM
Aug 11 22:31:12 dist-test-slave-8hhm krb5kdc[30657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487472, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.188@KRBTEST.COM
I20260811 22:31:12.627168 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.609699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:52155 (local address 127.18.211.189:39809)
0811 22:31:12.609832 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.609838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.611051 (+ 1213us) server_negotiation.cc:407] Connection header received
0811 22:31:12.612348 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.612352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.612409 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.612464 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.614003 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.614816 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.616085 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.616430 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.616531 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.621259 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.621263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.621266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.621298 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.623777 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.624634 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.624637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.624639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.624733 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.625334 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.625337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.625339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.625550 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.625636 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.626905 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.627019 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:31:12.628093 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.611294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.18.211.131:46983)
0811 22:31:12.611583 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.611599 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.612105 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.612682 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.612689 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.613201 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.613832 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.613847 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.614707 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.614711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.616068 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.616079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.616805 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.616809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.617199 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.617964 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.617988 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.621795 (+ 3807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.623937 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.623945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.623962 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.624514 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.624837 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.624840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.624842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.625084 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.625630 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.625637 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.626688 (+ 1051us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.626790 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.626970 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":1047,"spinlock_wait_cycles":17280,"thread_start_us":105,"threads_started":1}
I20260811 22:31:12.628445 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.610189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:50725)
0811 22:31:12.611102 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.611114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.612097 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.612692 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.612695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.613268 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.613848 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.613857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.615431 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.615435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.616408 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.616423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.617399 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.617404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.617552 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.618111 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.618129 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.621413 (+ 3284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.625584 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.625588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.625590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.625856 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.626241 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.626247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.626252 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.626361 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.626944 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.626964 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.627088 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.627215 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.627372 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":143,"spinlock_wait_cycles":66816,"thread_start_us":157,"threads_started":1}
I20260811 22:31:12.629033 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.609690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:50725 (local address 127.18.211.190:39121)
0811 22:31:12.609832 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.609838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.611200 (+ 1362us) server_negotiation.cc:407] Connection header received
0811 22:31:12.612348 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.612352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.612409 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.612464 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.614003 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.614819 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.616574 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.616909 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.617025 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.622102 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.622107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.622109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.622138 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.625457 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.625996 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.626000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.626004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.626106 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.626515 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.626519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.626521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.626772 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.626888 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.628675 (+ 1787us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.628827 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:31:12.629148 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.611659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:46983 (local address 127.18.211.188:45005)
0811 22:31:12.611835 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.611839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.611855 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:12.612332 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.612337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.612404 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.612491 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.613998 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.614550 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.616206 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.616551 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.616657 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.621963 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.621968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.621971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.622005 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.623777 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.624634 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.624637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.624639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.624729 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.625225 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.625232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.625237 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.625471 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.625570 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.628851 (+ 3281us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.628999 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:31:12.629266 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.610189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.18.211.131:52155)
0811 22:31:12.610834 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.610922 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.612097 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.612682 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.612692 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.613267 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:12.613848 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.613857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.614935 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.614939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.615954 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.615963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.616805 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.616809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.617199 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.618240 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:12.618260 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:12.620983 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:12.623965 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.623969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.623970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.624514 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.624837 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:12.624840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:12.624842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:12.625084 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:12.625673 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:12.625676 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:12.626012 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.626552 (+ 540us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.629052 (+ 2500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":2465,"spinlock_wait_cycles":37760,"thread_start_us":161,"threads_started":1}
I20260811 22:31:12.630167 424 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39121
I20260811 22:31:12.630420 424 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.630499 427 heartbeater.cc:347] Connected to a master server at 127.18.211.188:45005
I20260811 22:31:12.630525 428 heartbeater.cc:347] Connected to a master server at 127.18.211.189:39809
I20260811 22:31:12.630574 427 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.630622 428 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:12.631074 428 heartbeater.cc:511] Master 127.18.211.189:39809 requested a full tablet report, sending...
I20260811 22:31:12.631209 424 heartbeater.cc:511] Master 127.18.211.190:39121 requested a full tablet report, sending...
I20260811 22:31:12.632266 30730 ts_manager.cc:194] Re-registered known tserver with Master: c4361fe589634297aeca20acc858faf0 (127.18.211.131:42535)
I20260811 22:31:12.632277 31780 ts_manager.cc:194] Registered new tserver with Master: c4361fe589634297aeca20acc858faf0 (127.18.211.131:42535)
I20260811 22:31:12.633116 30730 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:50725
I20260811 22:31:12.631079 427 heartbeater.cc:511] Master 127.18.211.188:45005 requested a full tablet report, sending...
I20260811 22:31:12.633970 32129 ts_manager.cc:194] Registered new tserver with Master: c4361fe589634297aeca20acc858faf0 (127.18.211.131:42535)
I20260811 22:31:12.636116 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.619472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.18.211.190:39121)
0811 22:31:12.619852 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.619856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.619873 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:31:12.619920 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.619923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.619973 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.620073 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.621132 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.621937 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.624112 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.624473 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.624599 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.632049 (+ 7450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.632053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.632056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.632082 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.633972 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.634518 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.634522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.634524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.634628 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.635001 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.635005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.635007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.635228 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.635367 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.635766 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.635907 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":104,"threads_started":1}
I20260811 22:31:12.637522 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.620234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49822 (local address 127.18.211.188:45005)
0811 22:31:12.620871 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.620878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.622002 (+ 1124us) server_negotiation.cc:407] Connection header received
0811 22:31:12.622055 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.622059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.622113 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.622204 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.623222 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.624037 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.625274 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.625667 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.625780 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.627629 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.627634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.627637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.627664 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.635352 (+ 7688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.635879 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.635883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.635884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.635970 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.636300 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.636306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.636311 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.636537 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.636668 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.636955 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.637086 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":100,"threads_started":1}
I20260811 22:31:12.638274 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.619645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.18.211.189:39809)
0811 22:31:12.622375 (+ 2730us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.622381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.622429 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:31:12.622467 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.622473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.622901 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.629525 (+ 6624us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.629631 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.630511 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.631662 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.631992 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.632429 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.633785 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.633789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.633791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.633814 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.635621 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.636136 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.636139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.636141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.636229 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.636782 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.636784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.636785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.637037 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.637866 (+ 829us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.637950 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.638090 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2645,"thread_start_us":99,"threads_started":1}
I20260811 22:31:12.639380 30730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:31:12.642812 30730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:12.645517 32060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: NON_VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: true } } }
I20260811 22:31:12.645650 32060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:12.649344 453 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: attempting to promote NON_VOTER 744bd6308ad54d5286826c5b4f01ca34 to VOTER
I20260811 22:31:12.649708 453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } }
I20260811 22:31:12.651039 31795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Refusing update from remote peer 545da267b67449a687813d8a1e5680f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:31:12.651494 450 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:31:12.651711 32150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 LEARNER]: Refusing update from remote peer 545da267b67449a687813d8a1e5680f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:31:12.652118 450 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:31:12.653784 32060 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.654270 31795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.654718 32150 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.655205 452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.655230 32060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.655313 452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:12.655340 32060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:12.655462 452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.655570 452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:12.656508 463 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.656633 463 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:31:12.659929 32201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "545da267b67449a687813d8a1e5680f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:12.660075 32201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:31:12.664566 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.658749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:42535 (local address 127.0.0.1:33206)
0811 22:31:12.658907 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.658922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.658999 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.659824 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.659827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.659851 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:12.659961 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.659969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.662141 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.662144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.663318 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.663328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.664197 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.664200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.664284 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.664306 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.664366 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.664439 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:31:12.664639 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.658928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.18.211.131:42535)
0811 22:31:12.659265 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.659275 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.659295 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:31:12.659363 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.659367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.659642 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.659829 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:12.660084 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.662022 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.663439 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.664056 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.664198 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.664343 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.664409 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.664501 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":140,"threads_started":1}
I20260811 22:31:12.666530 478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.660407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:35859 (local address 127.0.0.1:40320)
0811 22:31:12.661026 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.661041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.661155 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.661661 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.661664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.661691 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:12.661827 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.661834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.663215 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.663219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.664907 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.664916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.666142 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.666145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.666236 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.666259 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.666330 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.666395 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":121,"threads_started":1}
I20260811 22:31:12.667073 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.660562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40320 (local address 127.18.211.130:35859)
0811 22:31:12.661003 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.661016 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.661147 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:31:12.661304 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.661309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.661531 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.661674 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:12.661927 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.663067 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.665036 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.665795 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.665940 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.666075 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.666780 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.666932 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":111,"threads_started":1}
I20260811 22:31:12.667377 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.661007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.18.211.129:38915)
0811 22:31:12.662303 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.662316 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.662333 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:31:12.662446 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.662453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.662618 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.662808 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:12.662997 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.664132 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.665901 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.666756 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.666871 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.666994 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.667130 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.667235 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1206,"thread_start_us":100,"threads_started":1}
I20260811 22:31:12.667692 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.660797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38915 (local address 127.0.0.1:36992)
0811 22:31:12.661362 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.661375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.661457 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.662781 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.662784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.662803 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:12.662898 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.662904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.664771 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.664774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.665761 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.665772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.666892 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.666895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.666985 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.667012 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.667097 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.667318 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":94,"threads_started":1}
I20260811 22:31:12.668149 318 tablet_service.cc:1467] Processing CreateTablet for tablet 628384f1d3dd42b1a2af344cfa77b5d4 (DEFAULT_TABLE table=test-table [id=3ad2bb4d07c74fa889d514a8661092bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:12.669430 318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628384f1d3dd42b1a2af344cfa77b5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:12.670109 32562 tablet_service.cc:1467] Processing CreateTablet for tablet 628384f1d3dd42b1a2af344cfa77b5d4 (DEFAULT_TABLE table=test-table [id=3ad2bb4d07c74fa889d514a8661092bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:12.670557 32317 tablet_service.cc:1467] Processing CreateTablet for tablet 628384f1d3dd42b1a2af344cfa77b5d4 (DEFAULT_TABLE table=test-table [id=3ad2bb4d07c74fa889d514a8661092bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:12.671178 32562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628384f1d3dd42b1a2af344cfa77b5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:12.671546 32317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628384f1d3dd42b1a2af344cfa77b5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:12.676725 481 tablet_bootstrap.cc:492] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Bootstrap starting.
I20260811 22:31:12.678787 481 tablet_bootstrap.cc:654] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:12.679265 482 tablet_bootstrap.cc:492] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Bootstrap starting.
I20260811 22:31:12.680205 483 tablet_bootstrap.cc:492] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Bootstrap starting.
I20260811 22:31:12.680989 481 log.cc:824] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:12.681663 482 tablet_bootstrap.cc:654] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:12.682216 483 tablet_bootstrap.cc:654] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:12.682477 482 log.cc:824] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:12.683084 483 log.cc:824] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:12.683300 481 tablet_bootstrap.cc:492] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: No bootstrap required, opened a new log
I20260811 22:31:12.683542 481 ts_tablet_manager.cc:1397] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:31:12.684741 482 tablet_bootstrap.cc:492] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: No bootstrap required, opened a new log
I20260811 22:31:12.684931 482 ts_tablet_manager.cc:1397] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:31:12.686725 483 tablet_bootstrap.cc:492] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: No bootstrap required, opened a new log
I20260811 22:31:12.686909 483 ts_tablet_manager.cc:1397] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260811 22:31:12.688437 481 raft_consensus.cc:348] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.688725 481 raft_consensus.cc:374] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:12.688786 481 raft_consensus.cc:729] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4361fe589634297aeca20acc858faf0, State: Initialized, Role: FOLLOWER
I20260811 22:31:12.688894 482 raft_consensus.cc:348] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.689131 482 raft_consensus.cc:374] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:12.689196 482 raft_consensus.cc:729] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3cdc0758af94b74b610e3093a7b8a5a, State: Initialized, Role: FOLLOWER
I20260811 22:31:12.689314 481 consensus_queue.cc:260] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.689677 482 consensus_queue.cc:260] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.690335 481 ts_tablet_manager.cc:1428] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:31:12.690366 32437 heartbeater.cc:503] Master 127.18.211.190:39121 was elected leader, sending a full tablet report...
I20260811 22:31:12.690259 483 raft_consensus.cc:348] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.690510 483 raft_consensus.cc:374] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:12.690557 483 raft_consensus.cc:729] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93a01bbb5e8f4acdbdd519dcc69ca548, State: Initialized, Role: FOLLOWER
I20260811 22:31:12.690670 424 heartbeater.cc:503] Master 127.18.211.190:39121 was elected leader, sending a full tablet report...
I20260811 22:31:12.690976 483 consensus_queue.cc:260] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.691944 483 ts_tablet_manager.cc:1428] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:31:12.692268 32669 heartbeater.cc:503] Master 127.18.211.190:39121 was elected leader, sending a full tablet report...
I20260811 22:31:12.696102 482 ts_tablet_manager.cc:1428] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260811 22:31:12.766093 498 raft_consensus.cc:482] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:12.766314 498 raft_consensus.cc:504] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.768028 498 leader_election.cc:288] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4361fe589634297aeca20acc858faf0 (127.18.211.131:42535), e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:12.773036 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.768535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:60843 (local address 127.18.211.129:38915)
0811 22:31:12.768652 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.768656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.768669 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:12.768732 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.768737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.768786 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.768858 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:12.769101 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.769937 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.771749 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.772362 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.772491 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.772548 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.772825 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.772901 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:31:12.773036 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.768369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:40051 (local address 127.18.211.131:42535)
0811 22:31:12.768510 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.768515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.768532 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:31:12.768620 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.768623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.768680 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.768765 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:12.769101 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.769918 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.771749 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.772365 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.772491 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.772548 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.772825 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.772903 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:31:12.773044 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.768353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38915 (local address 127.18.211.130:60843)
0811 22:31:12.768538 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.768553 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.768616 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.768880 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.768882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.768894 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:12.768979 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.768983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.770045 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.770049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.771593 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.771606 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.772492 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.772496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.772678 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.772697 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.772780 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.772885 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":194}
I20260811 22:31:12.773044 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.768224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:42535 (local address 127.18.211.130:40051)
0811 22:31:12.768369 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:12.768388 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:12.768511 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:12.768791 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:12.768793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:12.768809 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:12.768968 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.768974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.770045 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.770049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.771595 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:12.771606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.772492 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.772496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:12.772664 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.772698 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:31:12.772780 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:31:12.772891 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":230}
I20260811 22:31:12.773852 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628384f1d3dd42b1a2af344cfa77b5d4" candidate_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" is_pre_election: true
I20260811 22:31:12.774168 351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628384f1d3dd42b1a2af344cfa77b5d4" candidate_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4361fe589634297aeca20acc858faf0" is_pre_election: true
I20260811 22:31:12.774228 32359 raft_consensus.cc:2454] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93a01bbb5e8f4acdbdd519dcc69ca548 in term 0.
I20260811 22:31:12.774501 351 raft_consensus.cc:2454] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93a01bbb5e8f4acdbdd519dcc69ca548 in term 0.
I20260811 22:31:12.774787 32486 leader_election.cc:302] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93a01bbb5e8f4acdbdd519dcc69ca548, e3cdc0758af94b74b610e3093a7b8a5a; no voters:
I20260811 22:31:12.775134 498 raft_consensus.cc:2781] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:12.775228 498 raft_consensus.cc:482] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:12.775275 498 raft_consensus.cc:3037] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:12.776434 498 raft_consensus.cc:504] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.776779 498 leader_election.cc:288] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [CANDIDATE]: Term 1 election: Requested vote from peers c4361fe589634297aeca20acc858faf0 (127.18.211.131:42535), e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:12.777114 351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628384f1d3dd42b1a2af344cfa77b5d4" candidate_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4361fe589634297aeca20acc858faf0"
I20260811 22:31:12.777199 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628384f1d3dd42b1a2af344cfa77b5d4" candidate_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3cdc0758af94b74b610e3093a7b8a5a"
I20260811 22:31:12.777261 351 raft_consensus.cc:3037] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:12.777388 32359 raft_consensus.cc:3037] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:12.778522 32359 raft_consensus.cc:2454] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93a01bbb5e8f4acdbdd519dcc69ca548 in term 1.
I20260811 22:31:12.778673 351 raft_consensus.cc:2454] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93a01bbb5e8f4acdbdd519dcc69ca548 in term 1.
I20260811 22:31:12.779016 32480 leader_election.cc:302] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93a01bbb5e8f4acdbdd519dcc69ca548, c4361fe589634297aeca20acc858faf0; no voters:
I20260811 22:31:12.779214 498 raft_consensus.cc:2781] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:12.779456 498 raft_consensus.cc:686] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 LEADER]: Becoming Leader. State: Replica: 93a01bbb5e8f4acdbdd519dcc69ca548, State: Running, Role: LEADER
I20260811 22:31:12.780059 498 consensus_queue.cc:237] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } }
I20260811 22:31:12.783742 30726 catalog_manager.cc:5495] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130). New cstate: current_term: 1 leader_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } health_report { overall_health: HEALTHY } } }
I20260811 22:31:12.817644 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.813885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.18.211.130:35859)
0811 22:31:12.814043 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.814048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.814064 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:12.814242 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.814245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.814304 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.814428 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:12.814697 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.815206 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.816074 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.816292 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.816403 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.816749 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:12.817236 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:12.817375 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.817418 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.817477 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:12.823815 351 raft_consensus.cc:1261] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 1 FOLLOWER]: Refusing update from remote peer 93a01bbb5e8f4acdbdd519dcc69ca548: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:12.823815 32359 raft_consensus.cc:1261] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Refusing update from remote peer 93a01bbb5e8f4acdbdd519dcc69ca548: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:12.824604 498 consensus_queue.cc:1035] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:12.824841 499 consensus_queue.cc:1035] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:12.843814 30726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:12.843990 30726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:12.848944 30726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 545da267b67449a687813d8a1e5680f4: Sending DeleteTablet for 3 replicas of tablet 628384f1d3dd42b1a2af344cfa77b5d4
I20260811 22:31:12.849723 318 tablet_service.cc:1514] Processing DeleteTablet for tablet 628384f1d3dd42b1a2af344cfa77b5d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:12 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:33206
I20260811 22:31:12.849941 32562 tablet_service.cc:1514] Processing DeleteTablet for tablet 628384f1d3dd42b1a2af344cfa77b5d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:12 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:40320
I20260811 22:31:12.850482 522 tablet_replica.cc:330] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: stopping tablet replica
I20260811 22:31:12.850829 522 raft_consensus.cc:2229] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:12.851203 522 raft_consensus.cc:2258] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:12.852532 32317 tablet_service.cc:1514] Processing DeleteTablet for tablet 628384f1d3dd42b1a2af344cfa77b5d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:12 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:36992
I20260811 22:31:12.852921 522 ts_tablet_manager.cc:1905] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:12.853107 525 tablet_replica.cc:330] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: stopping tablet replica
I20260811 22:31:12.853421 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 30676
I20260811 22:31:12.853575 525 raft_consensus.cc:2229] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:12.864820 522 ts_tablet_manager.cc:1918] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:12.864961 522 log.cc:1196] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/628384f1d3dd42b1a2af344cfa77b5d4
I20260811 22:31:12.864949 525 raft_consensus.cc:2258] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:12.865338 520 tablet_replica.cc:330] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: stopping tablet replica
I20260811 22:31:12.865371 522 ts_tablet_manager.cc:1939] T 628384f1d3dd42b1a2af344cfa77b5d4 P 93a01bbb5e8f4acdbdd519dcc69ca548: Deleting consensus metadata
I20260811 22:31:12.865798 520 raft_consensus.cc:2229] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:12.866266 520 raft_consensus.cc:2258] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:12.870721 525 ts_tablet_manager.cc:1905] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:12.873726 525 ts_tablet_manager.cc:1918] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:12.873842 525 log.cc:1196] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/628384f1d3dd42b1a2af344cfa77b5d4
I20260811 22:31:12.874205 525 ts_tablet_manager.cc:1939] T 628384f1d3dd42b1a2af344cfa77b5d4 P e3cdc0758af94b74b610e3093a7b8a5a: Deleting consensus metadata
I20260811 22:31:12.875036 520 ts_tablet_manager.cc:1905] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:31:12.878177 32250 connection.cc:535] server connection from 127.0.0.1:36992 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 22:31:12.878199 32486 connection.cc:535] server connection from 127.0.0.1:40320 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:31:12.878639 520 ts_tablet_manager.cc:1918] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:12.878762 520 log.cc:1196] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/628384f1d3dd42b1a2af344cfa77b5d4
I20260811 22:31:12.879148 520 ts_tablet_manager.cc:1939] T 628384f1d3dd42b1a2af344cfa77b5d4 P c4361fe589634297aeca20acc858faf0: Deleting consensus metadata
W20260811 22:31:12.880388 32712 connection.cc:441] server connection from 127.0.0.1:33206 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33206 (request call id 1) could send its response
I20260811 22:31:12.900019 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.884018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.18.211.188:45005)
0811 22:31:12.884135 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.884151 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.886936 (+ 2785us) server_negotiation.cc:407] Connection header received
0811 22:31:12.886994 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.886997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.887055 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.887127 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.889826 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.890598 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.891662 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.891986 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.892102 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.895887 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.895891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.895894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.895924 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.898097 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.898607 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.898611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.898613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.898700 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.899005 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.899009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.899010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.899249 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.899356 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.899719 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.899845 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:31:12.900745 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.851015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.18.211.189:39809)
0811 22:31:12.851158 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.851165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.882964 (+ 31799us) server_negotiation.cc:407] Connection header received
0811 22:31:12.883207 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.883211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.883286 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.883353 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.884576 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.885453 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.887543 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.888594 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.888707 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.895839 (+ 7132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.895843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.895846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.895888 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.898394 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.899016 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.899020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.899022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.899146 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.899477 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.899480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.899482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.899695 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.899832 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.900419 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.900568 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43,"spinlock_wait_cycles":14976}
I20260811 22:31:12.901331 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.883381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.18.211.189:39809)
0811 22:31:12.883496 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.883500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.885019 (+ 1519us) server_negotiation.cc:407] Connection header received
0811 22:31:12.885329 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.885338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.885391 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.885452 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.888546 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.889340 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.890555 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.890875 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.890948 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.895839 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.895843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.895846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.895888 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.898401 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.899016 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.899020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.899022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.899140 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.899477 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.899480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.899482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.900048 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.900154 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.900494 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.901179 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":569,"server-negotiator.queue_time_us":44,"spinlock_wait_cycles":11392}
I20260811 22:31:12.909525 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:12.851538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.18.211.188:45005)
0811 22:31:12.851678 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:12.851690 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:12.901655 (+ 49965us) server_negotiation.cc:407] Connection header received
0811 22:31:12.901828 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:12.901832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:12.901878 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:12.901975 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:12.902875 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.903392 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.904190 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:12.904426 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:12.904533 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:12.905999 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:12.906003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:12.906006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:12.906032 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:12.907878 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.908337 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.908340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.908341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.908414 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:12.908699 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:12.908701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:12.908703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:12.908859 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:12.908955 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:12.909189 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:31:12.909351 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:31:13.838039 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:13.837436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.129:54859)
0811 22:31:13.837787 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:13.837901 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
W20260811 22:31:13.838397 32437 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:39121 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
I20260811 22:31:13.842104 543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:13.841499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:49213)
0811 22:31:13.841856 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:13.841972 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":85,"threads_started":1}
W20260811 22:31:13.842465 424 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:39121 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
I20260811 22:31:13.842919 544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:13.842174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.130:39495)
0811 22:31:13.842723 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:13.842818 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":74,"threads_started":1}
W20260811 22:31:13.843178 32669 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:39121 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
I20260811 22:31:14.348534 546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 545da267b67449a687813d8a1e5680f4)
I20260811 22:31:14.348752 546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } }
I20260811 22:31:14.349190 549 raft_consensus.cc:482] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 545da267b67449a687813d8a1e5680f4)
I20260811 22:31:14.349478 549 raft_consensus.cc:504] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } }
I20260811 22:31:14.350101 546 leader_election.cc:288] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 545da267b67449a687813d8a1e5680f4 (127.18.211.190:39121), df847b860e2f47a0bccdb34d0b30f30f (127.18.211.189:39809)
I20260811 22:31:14.350625 549 leader_election.cc:288] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 545da267b67449a687813d8a1e5680f4 (127.18.211.190:39121), 744bd6308ad54d5286826c5b4f01ca34 (127.18.211.188:45005)
I20260811 22:31:14.350976 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.350322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52988)
0811 22:31:14.350668 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.350864 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":74,"threads_started":1}
I20260811 22:31:14.351290 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.350707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:52994)
0811 22:31:14.351086 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.351161 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":77,"threads_started":1}
W20260811 22:31:14.351388 32102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
W20260811 22:31:14.351462 31756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
I20260811 22:31:14.352452 565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.352041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:53008)
0811 22:31:14.352280 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.352341 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":95,"threads_started":1}
I20260811 22:31:14.352790 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.352557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:53022)
0811 22:31:14.352686 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.352730 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 22:31:14.352943 31756 leader_election.cc:334] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 545da267b67449a687813d8a1e5680f4 (127.18.211.190:39121): Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
W20260811 22:31:14.353163 32102 leader_election.cc:334] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 545da267b67449a687813d8a1e5680f4 (127.18.211.190:39121): Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
I20260811 22:31:14.355502 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.350627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.18.211.189:39809)
0811 22:31:14.350910 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.350915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.351172 (+ 257us) server_negotiation.cc:407] Connection header received
0811 22:31:14.351418 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.351422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.351481 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.351548 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:14.352007 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.352957 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.354433 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.355014 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.355127 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.355169 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.355363 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.355419 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":65,"threads_started":1}
I20260811 22:31:14.355552 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.350415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:39809 (local address 127.0.0.1:36320)
0811 22:31:14.351083 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.351099 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:14.351270 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:14.351755 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:14.351758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:14.351780 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:14.351893 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.351901 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.353152 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.353156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.354307 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.354317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.355131 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.355135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.355228 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.355257 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:31:14.355343 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:31:14.355415 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":471,"threads_started":1}
I20260811 22:31:14.356117 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "744bd6308ad54d5286826c5b4f01ca34" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "df847b860e2f47a0bccdb34d0b30f30f" is_pre_election: true
I20260811 22:31:14.356353 31795 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744bd6308ad54d5286826c5b4f01ca34 in term 1.
I20260811 22:31:14.356500 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.351892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:45005 (local address 127.0.0.1:49840)
0811 22:31:14.352055 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.352068 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:14.352168 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:14.352598 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:14.352601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:14.352623 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:14.352734 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.352741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.353904 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.353906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.354933 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.354943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.356063 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.356066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.356158 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.356192 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:31:14.356273 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:31:14.356343 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":1}
I20260811 22:31:14.356542 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.351124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49840 (local address 127.18.211.188:45005)
0811 22:31:14.351436 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.351441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.352127 (+ 686us) server_negotiation.cc:407] Connection header received
0811 22:31:14.352296 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.352299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.352349 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.352433 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:14.352874 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.353763 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.355094 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.355958 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.356055 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.356092 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.356314 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.356399 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260811 22:31:14.356818 32109 leader_election.cc:302] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 744bd6308ad54d5286826c5b4f01ca34, df847b860e2f47a0bccdb34d0b30f30f; no voters: 545da267b67449a687813d8a1e5680f4
I20260811 22:31:14.357120 546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:31:14.357201 546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 545da267b67449a687813d8a1e5680f4)
I20260811 22:31:14.357300 546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:31:14.357738 32150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df847b860e2f47a0bccdb34d0b30f30f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "744bd6308ad54d5286826c5b4f01ca34" is_pre_election: true
I20260811 22:31:14.358639 546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } }
I20260811 22:31:14.358919 32150 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df847b860e2f47a0bccdb34d0b30f30f in current term 2: Already voted for candidate 744bd6308ad54d5286826c5b4f01ca34 in this term.
I20260811 22:31:14.359119 546 leader_election.cc:288] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [CANDIDATE]: Term 2 election: Requested vote from peers 545da267b67449a687813d8a1e5680f4 (127.18.211.190:39121), df847b860e2f47a0bccdb34d0b30f30f (127.18.211.189:39809)
I20260811 22:31:14.359433 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.359206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:53046)
0811 22:31:14.359297 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.359360 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 22:31:14.359505 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "744bd6308ad54d5286826c5b4f01ca34" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "df847b860e2f47a0bccdb34d0b30f30f"
I20260811 22:31:14.359628 31795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:31:14.360234 31760 leader_election.cc:302] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df847b860e2f47a0bccdb34d0b30f30f; no voters: 545da267b67449a687813d8a1e5680f4, 744bd6308ad54d5286826c5b4f01ca34
I20260811 22:31:14.360751 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.360434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:53052)
0811 22:31:14.360625 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.360683 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 22:31:14.360867 32102 leader_election.cc:334] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 545da267b67449a687813d8a1e5680f4 (127.18.211.190:39121): Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
I20260811 22:31:14.360880 31795 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744bd6308ad54d5286826c5b4f01ca34 in term 2.
I20260811 22:31:14.361020 549 raft_consensus.cc:2726] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:31:14.361152 32109 leader_election.cc:302] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 744bd6308ad54d5286826c5b4f01ca34, df847b860e2f47a0bccdb34d0b30f30f; no voters: 545da267b67449a687813d8a1e5680f4
I20260811 22:31:14.361339 546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:31:14.361588 546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [term 2 LEADER]: Becoming Leader. State: Replica: 744bd6308ad54d5286826c5b4f01ca34, State: Running, Role: LEADER
I20260811 22:31:14.361852 546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } }
I20260811 22:31:14.363160 568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 744bd6308ad54d5286826c5b4f01ca34. Latest consensus state: current_term: 2 leader_uuid: "744bd6308ad54d5286826c5b4f01ca34" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:14.363301 568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:14.363624 571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:14.366230 571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:14.366679 571 catalog_manager.cc:1259] Loaded cluster ID: 72d2d6dc099a40d1bc2fab835e24ead2
I20260811 22:31:14.366727 571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:14.367383 571 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:14.367831 571 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Loaded TSK: 0
I20260811 22:31:14.368631 571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:31:14.417582 32128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:31:14.420768 32128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:14.423995 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.423672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:53062)
0811 22:31:14.423821 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.423896 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:31:14.424069 31795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [term 2 FOLLOWER]: Refusing update from remote peer 744bd6308ad54d5286826c5b4f01ca34: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 22:31:14.424639 546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 22:31:14.424851 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.424638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.0.0.1:53070)
0811 22:31:14.424727 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.424781 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 22:31:14.424981 32102 consensus_peers.cc:487] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 -> Peer 545da267b67449a687813d8a1e5680f4 (127.18.211.190:39121): Couldn't send request to peer 545da267b67449a687813d8a1e5680f4. Status: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:31:14.426276 549 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 744bd6308ad54d5286826c5b4f01ca34. Latest consensus state: current_term: 2 leader_uuid: "744bd6308ad54d5286826c5b4f01ca34" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:14.426445 549 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:31:14.427736 568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "744bd6308ad54d5286826c5b4f01ca34" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:14.427862 568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:14.427991 549 sys_catalog.cc:455] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "744bd6308ad54d5286826c5b4f01ca34" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "545da267b67449a687813d8a1e5680f4" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39121 } } peers { permanent_uuid: "df847b860e2f47a0bccdb34d0b30f30f" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 39809 } attrs { promote: false } } peers { permanent_uuid: "744bd6308ad54d5286826c5b4f01ca34" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 45005 } attrs { promote: false } } }
I20260811 22:31:14.428115 549 sys_catalog.cc:458] T 00000000000000000000000000000000 P df847b860e2f47a0bccdb34d0b30f30f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:31:14.438499 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.434201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:42535 (local address 127.0.0.1:33222)
0811 22:31:14.434465 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.434482 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:14.434565 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:14.435017 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:14.435020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:14.435036 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:14.435129 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.435133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.436143 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.436148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.437351 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.437360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.438091 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.438095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.438183 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.438213 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:31:14.438263 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 22:31:14.438337 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":21}
I20260811 22:31:14.439116 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.434475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.18.211.131:42535)
0811 22:31:14.434759 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.434763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.434777 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:14.434834 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.434836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.434875 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.434990 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:14.435245 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.436000 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.437471 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.437935 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.438755 (+ 820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.438817 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.438859 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.438949 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
I20260811 22:31:14.439409 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.434525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:35859 (local address 127.0.0.1:40332)
0811 22:31:14.434638 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.434653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:14.434754 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:14.435656 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:14.435660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:14.435680 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:14.435766 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.435771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.436828 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.436831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.437856 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.437866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.438969 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.438973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.439064 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.439093 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:31:14.439185 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:31:14.439280 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":80}
I20260811 22:31:14.439663 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.435065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38915 (local address 127.0.0.1:37000)
0811 22:31:14.435333 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.435347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:14.435433 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:14.435933 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:14.435935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:14.435948 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:14.436017 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.436022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.436941 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.436944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.438437 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.438442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.439284 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.439288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.439371 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.439393 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:31:14.439445 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 22:31:14.439516 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":655,"thread_start_us":118,"threads_started":1}
I20260811 22:31:14.439684 318 tablet_service.cc:1467] Processing CreateTablet for tablet 2ec3d70a22184dd395f6a96ddbebf5b0 (DEFAULT_TABLE table=test-table [id=40c5fdf2a2614480aca996b434fb59a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:14.440053 318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ec3d70a22184dd395f6a96ddbebf5b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:14.440230 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.435439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.18.211.129:38915)
0811 22:31:14.435693 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.435697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.435713 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:14.435765 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.435768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.435815 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.435931 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:14.436101 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.436829 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.438559 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.439155 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.439875 (+ 720us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.439931 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.439982 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.440081 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260811 22:31:14.440758 32317 tablet_service.cc:1467] Processing CreateTablet for tablet 2ec3d70a22184dd395f6a96ddbebf5b0 (DEFAULT_TABLE table=test-table [id=40c5fdf2a2614480aca996b434fb59a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:14.441075 32317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ec3d70a22184dd395f6a96ddbebf5b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:14.441730 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.434832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40332 (local address 127.18.211.130:35859)
0811 22:31:14.435382 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.435388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.435404 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:14.435461 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.435466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.435516 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.435638 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:14.435902 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.436693 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.437987 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.438830 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.438963 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.439011 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.439385 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.441602 (+ 2217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":110,"threads_started":1}
I20260811 22:31:14.442394 32562 tablet_service.cc:1467] Processing CreateTablet for tablet 2ec3d70a22184dd395f6a96ddbebf5b0 (DEFAULT_TABLE table=test-table [id=40c5fdf2a2614480aca996b434fb59a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:14.442701 32562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ec3d70a22184dd395f6a96ddbebf5b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:14.442968 593 tablet_bootstrap.cc:492] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: Bootstrap starting.
I20260811 22:31:14.444146 594 tablet_bootstrap.cc:492] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a: Bootstrap starting.
I20260811 22:31:14.444998 597 tablet_bootstrap.cc:492] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: Bootstrap starting.
I20260811 22:31:14.445178 594 tablet_bootstrap.cc:654] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:14.445181 593 tablet_bootstrap.cc:654] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:14.446252 597 tablet_bootstrap.cc:654] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:14.446530 594 tablet_bootstrap.cc:492] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a: No bootstrap required, opened a new log
I20260811 22:31:14.446664 594 ts_tablet_manager.cc:1397] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:14.446867 593 tablet_bootstrap.cc:492] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: No bootstrap required, opened a new log
I20260811 22:31:14.446961 593 ts_tablet_manager.cc:1397] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:31:14.447176 594 raft_consensus.cc:348] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.447309 594 raft_consensus.cc:374] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:14.447347 594 raft_consensus.cc:729] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3cdc0758af94b74b610e3093a7b8a5a, State: Initialized, Role: FOLLOWER
I20260811 22:31:14.447364 593 raft_consensus.cc:348] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.447448 597 tablet_bootstrap.cc:492] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: No bootstrap required, opened a new log
I20260811 22:31:14.447485 593 raft_consensus.cc:374] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:14.447520 593 raft_consensus.cc:729] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4361fe589634297aeca20acc858faf0, State: Initialized, Role: FOLLOWER
I20260811 22:31:14.447567 597 ts_tablet_manager.cc:1397] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:14.447530 594 consensus_queue.cc:260] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.447639 593 consensus_queue.cc:260] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.447904 594 ts_tablet_manager.cc:1428] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:31:14.447980 593 ts_tablet_manager.cc:1428] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:14.448022 597 raft_consensus.cc:348] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.448164 597 raft_consensus.cc:374] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:14.448217 597 raft_consensus.cc:729] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93a01bbb5e8f4acdbdd519dcc69ca548, State: Initialized, Role: FOLLOWER
I20260811 22:31:14.448344 597 consensus_queue.cc:260] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.448681 597 ts_tablet_manager.cc:1428] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:14.450861 32438 heartbeater.cc:503] Master 127.18.211.188:45005 was elected leader, sending a full tablet report...
I20260811 22:31:14.451592 32670 heartbeater.cc:503] Master 127.18.211.188:45005 was elected leader, sending a full tablet report...
I20260811 22:31:14.452515 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.449075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.130:52799)
0811 22:31:14.452193 (+ 3118us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.452264 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3047,"thread_start_us":99,"threads_started":1}
I20260811 22:31:14.452592 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.448996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:43221)
0811 22:31:14.452387 (+ 3391us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.452450 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3322,"thread_start_us":97,"threads_started":1}
I20260811 22:31:14.453200 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.448304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.129:50737)
0811 22:31:14.452969 (+ 4665us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.453031 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4590,"thread_start_us":100,"threads_started":1}
I20260811 22:31:14.453290 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.452940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.130:57901)
0811 22:31:14.453113 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.453165 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":102,"threads_started":1}
I20260811 22:31:14.453682 427 heartbeater.cc:503] Master 127.18.211.188:45005 was elected leader, sending a full tablet report...
I20260811 22:31:14.454049 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.453815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:48959)
0811 22:31:14.453907 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.453954 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 22:31:14.454105 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.453658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.129:35555)
0811 22:31:14.453942 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.454001 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":98,"threads_started":1}
I20260811 22:31:14.490329 607 raft_consensus.cc:482] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:14.490484 607 raft_consensus.cc:504] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.491724 607 leader_election.cc:288] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130:35859), e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:14.495431 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.492041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:54737 (local address 127.18.211.130:35859)
0811 22:31:14.492184 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.492190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.492206 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:14.492350 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.492353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.492398 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.492485 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:14.492771 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.493353 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.494462 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.494855 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.494981 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.495037 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.495237 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.495310 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:31:14.495431 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.491865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:35859 (local address 127.18.211.131:54737)
0811 22:31:14.492077 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.492094 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:14.492208 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:14.492500 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:14.492503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:14.492522 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:14.492652 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.492657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.493464 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.493469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.494326 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.494336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.494979 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.494983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.495099 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.495118 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:31:14.495217 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:31:14.495290 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":22}
I20260811 22:31:14.495584 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.492070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:41659 (local address 127.18.211.129:38915)
0811 22:31:14.492227 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.492231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.492449 (+ 218us) server_negotiation.cc:407] Connection header received
0811 22:31:14.492606 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.492614 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.492661 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.492754 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:14.493018 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.493581 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.494634 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.495012 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.495144 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.495196 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.495378 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.495448 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:14.495620 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.491949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38915 (local address 127.18.211.131:41659)
0811 22:31:14.492340 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.492351 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:14.492481 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:14.492766 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:14.492769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:14.492785 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:14.492902 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.492908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.493697 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.493702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.494462 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:14.494471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.495138 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.495142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:14.495251 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.495274 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:31:14.495361 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:31:14.495432 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260811 22:31:14.496057 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec3d70a22184dd395f6a96ddbebf5b0" candidate_uuid: "c4361fe589634297aeca20acc858faf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" is_pre_election: true
I20260811 22:31:14.496133 32591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec3d70a22184dd395f6a96ddbebf5b0" candidate_uuid: "c4361fe589634297aeca20acc858faf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" is_pre_election: true
I20260811 22:31:14.496256 32359 raft_consensus.cc:2454] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4361fe589634297aeca20acc858faf0 in term 0.
I20260811 22:31:14.496419 32591 raft_consensus.cc:2454] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4361fe589634297aeca20acc858faf0 in term 0.
I20260811 22:31:14.496701 32716 leader_election.cc:302] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4361fe589634297aeca20acc858faf0, e3cdc0758af94b74b610e3093a7b8a5a; no voters:
I20260811 22:31:14.497012 607 raft_consensus.cc:2781] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:14.497080 607 raft_consensus.cc:482] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:14.497120 607 raft_consensus.cc:3037] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:14.498216 607 raft_consensus.cc:504] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.498564 607 leader_election.cc:288] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [CANDIDATE]: Term 1 election: Requested vote from peers 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130:35859), e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915)
I20260811 22:31:14.498906 32591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec3d70a22184dd395f6a96ddbebf5b0" candidate_uuid: "c4361fe589634297aeca20acc858faf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548"
I20260811 22:31:14.498950 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec3d70a22184dd395f6a96ddbebf5b0" candidate_uuid: "c4361fe589634297aeca20acc858faf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3cdc0758af94b74b610e3093a7b8a5a"
I20260811 22:31:14.499053 32591 raft_consensus.cc:3037] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:14.499114 32359 raft_consensus.cc:3037] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:14.500075 32591 raft_consensus.cc:2454] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4361fe589634297aeca20acc858faf0 in term 1.
I20260811 22:31:14.500093 32359 raft_consensus.cc:2454] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4361fe589634297aeca20acc858faf0 in term 1.
I20260811 22:31:14.500420 32716 leader_election.cc:302] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4361fe589634297aeca20acc858faf0, e3cdc0758af94b74b610e3093a7b8a5a; no voters:
I20260811 22:31:14.500608 607 raft_consensus.cc:2781] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:14.500865 607 raft_consensus.cc:686] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 1 LEADER]: Becoming Leader. State: Replica: c4361fe589634297aeca20acc858faf0, State: Running, Role: LEADER
I20260811 22:31:14.501222 607 consensus_queue.cc:237] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } }
I20260811 22:31:14.501686 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.501375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:42701)
0811 22:31:14.501546 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.501605 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 22:31:14.504237 32129 catalog_manager.cc:5495] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4361fe589634297aeca20acc858faf0 (127.18.211.131). New cstate: current_term: 1 leader_uuid: "c4361fe589634297aeca20acc858faf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4361fe589634297aeca20acc858faf0" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 } health_report { overall_health: UNKNOWN } } }
I20260811 22:31:14.520465 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.517163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.18.211.131:42535)
0811 22:31:14.517331 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:14.517335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:14.517349 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:14.517468 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:14.517470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:14.517524 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:14.517611 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:14.517842 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.518338 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.519147 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:14.519362 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:14.519435 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:14.519647 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:14.520090 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:14.520194 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:14.520241 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 22:31:14.520290 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:31:14.523310 32359 raft_consensus.cc:1261] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Refusing update from remote peer c4361fe589634297aeca20acc858faf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:14.523643 32591 raft_consensus.cc:1261] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 FOLLOWER]: Refusing update from remote peer c4361fe589634297aeca20acc858faf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:14.523912 625 consensus_queue.cc:1035] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3cdc0758af94b74b610e3093a7b8a5a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:14.524226 607 consensus_queue.cc:1035] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93a01bbb5e8f4acdbdd519dcc69ca548" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 35859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:14.525652 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.525352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.129:43779)
0811 22:31:14.525508 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.525570 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 22:31:14.526943 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.526387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.130:34353)
0811 22:31:14.526802 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.526867 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337}
I20260811 22:31:14.529771 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.529435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.130:56281)
0811 22:31:14.529599 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.529670 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 22:31:14.529779 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.528917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:34375)
0811 22:31:14.529572 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.529640 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 22:31:14.530615 629 mvcc.cc:205] Tried to move back new op lower bound from 7317452695642525696 to 7317452695560015872. Current Snapshot: MvccSnapshot[applied={T|T < 7317452695642525696}]
I20260811 22:31:14.530912 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.530642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.131:51511)
0811 22:31:14.530804 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.530853 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:31:14.533063 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.532808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39121 (local address 127.18.211.129:43601)
0811 22:31:14.532935 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.532993 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:39121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:31:14.539390 32128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:14.539587 32128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:14.543876 32128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 744bd6308ad54d5286826c5b4f01ca34: Sending DeleteTablet for 3 replicas of tablet 2ec3d70a22184dd395f6a96ddbebf5b0
I20260811 22:31:14.544691 32562 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ec3d70a22184dd395f6a96ddbebf5b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:14 UTC) from {username='oryx', principal='oryx/127.18.211.188@KRBTEST.COM'} at 127.0.0.1:40332
I20260811 22:31:14.544730 318 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ec3d70a22184dd395f6a96ddbebf5b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:14 UTC) from {username='oryx', principal='oryx/127.18.211.188@KRBTEST.COM'} at 127.0.0.1:33222
I20260811 22:31:14.544991 32317 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ec3d70a22184dd395f6a96ddbebf5b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:14 UTC) from {username='oryx', principal='oryx/127.18.211.188@KRBTEST.COM'} at 127.0.0.1:37000
I20260811 22:31:14.545471 651 tablet_replica.cc:330] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a: stopping tablet replica
I20260811 22:31:14.545665 651 raft_consensus.cc:2229] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:14.545897 651 raft_consensus.cc:2258] T 2ec3d70a22184dd395f6a96ddbebf5b0 P e3cdc0758af94b74b610e3093a7b8a5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:14.545917 648 tablet_replica.cc:330] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: stopping tablet replica
I20260811 22:31:14.546049 648 raft_consensus.cc:2229] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:14.546067 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 32057
I20260811 22:31:14.546993 649 tablet_replica.cc:330] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: stopping tablet replica
I20260811 22:31:14.547123 649 raft_consensus.cc:2229] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:14.547510 649 raft_consensus.cc:2258] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:14.546208 648 raft_consensus.cc:2258] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:14.555955 648 ts_tablet_manager.cc:1905] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:14.559558 649 ts_tablet_manager.cc:1905] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:14.563797 648 ts_tablet_manager.cc:1918] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:14.563884 648 log.cc:1196] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/2ec3d70a22184dd395f6a96ddbebf5b0
I20260811 22:31:14.564112 648 ts_tablet_manager.cc:1939] T 2ec3d70a22184dd395f6a96ddbebf5b0 P 93a01bbb5e8f4acdbdd519dcc69ca548: Deleting consensus metadata
I20260811 22:31:14.564991 32102 catalog_manager.cc:4841] TS 93a01bbb5e8f4acdbdd519dcc69ca548 (127.18.211.130:35859): tablet 2ec3d70a22184dd395f6a96ddbebf5b0 (table test-table [id=40c5fdf2a2614480aca996b434fb59a9]) successfully deleted
I20260811 22:31:14.566300 649 ts_tablet_manager.cc:1918] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:14.566380 649 log.cc:1196] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/2ec3d70a22184dd395f6a96ddbebf5b0
I20260811 22:31:14.566640 649 ts_tablet_manager.cc:1939] T 2ec3d70a22184dd395f6a96ddbebf5b0 P c4361fe589634297aeca20acc858faf0: Deleting consensus metadata
I20260811 22:31:14.567459 32102 catalog_manager.cc:4841] TS c4361fe589634297aeca20acc858faf0 (127.18.211.131:42535): tablet 2ec3d70a22184dd395f6a96ddbebf5b0 (table test-table [id=40c5fdf2a2614480aca996b434fb59a9]) successfully deleted
I20260811 22:31:14.572271 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 32449
I20260811 22:31:14.572324 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:14.571992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38915 (local address 127.0.0.1:37012)
0811 22:31:14.572137 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:14.572220 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:31:14.581820 32109 catalog_manager.cc:4584] TS e3cdc0758af94b74b610e3093a7b8a5a (127.18.211.129:38915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ec3d70a22184dd395f6a96ddbebf5b0: Network error: Client connection negotiation failed: client connection to 127.18.211.129:38915: connect: Connection refused (error 111)
I20260811 22:31:14.588299 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 32679
I20260811 22:31:14.605099 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 31532
I20260811 22:31:14.624346 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 31713
2026-08-11T22:31:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5802 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:31:14 dist-test-slave-8hhm krb5kdc[654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:31:14 dist-test-slave-8hhm krb5kdc[654](info): set up 2 sockets
Aug 11 22:31:14 dist-test-slave-8hhm krb5kdc[654](info): commencing operation
krb5kdc: starting...
W20260811 22:31:15.962544 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:16 dist-test-slave-8hhm krb5kdc[654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab.
Entry for principal oryx/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab.
I20260811 22:31:16.098483 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43271
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab
--principal=oryx/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:43271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:16.217514 676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:16.217891 676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:16.217971 676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:16.222265 676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:16.222376 676 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:31:16.222406 676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:16.222450 676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:16.222513 676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:16.222551 676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:16.228226 676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:43271
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.190.keytab
--principal=oryx/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:16.229609 676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:16.230741 676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:16.240036 683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:16.240067 690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:16.240037 684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:16.240387 676 server_base.cc:1034] running on GCE node
I20260811 22:31:16.241276 676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:16.242524 676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:16.243741 676 hybrid_clock.cc:648] HybridClock initialized: now 1786487476243724 us; error 31 us; skew 500 ppm
Aug 11 22:31:16 dist-test-slave-8hhm krb5kdc[654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:16.246867 676 init.cc:377] Logged in from keytab as oryx/127.18.211.190@KRBTEST.COM (short username oryx)
I20260811 22:31:16.247888 676 webserver.cc:460] Webserver started at http://127.18.211.190:42643/ using document root <none> and password file <none>
I20260811 22:31:16.248507 676 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:16.248606 676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:16.248860 676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:16.251098 676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "d19957c8e0874d0088e9b1c4ef5d828a"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "df48fd4bc35373f51eeb372bcaaae2cc"
server_key_iv: "bbe3e73fb5a1ec7eeffa2437ddd82b7b"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.251753 676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "d19957c8e0874d0088e9b1c4ef5d828a"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "df48fd4bc35373f51eeb372bcaaae2cc"
server_key_iv: "bbe3e73fb5a1ec7eeffa2437ddd82b7b"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.256045 676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:31:16.258896 699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:16.260260 676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:31:16.260432 676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d19957c8e0874d0088e9b1c4ef5d828a"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "df48fd4bc35373f51eeb372bcaaae2cc"
server_key_iv: "bbe3e73fb5a1ec7eeffa2437ddd82b7b"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.260565 676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:16.272655 676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:16.275938 676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:16.276185 676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:16.284868 676 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:43271
I20260811 22:31:16.284875 818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:43271 every 8 connection(s)
I20260811 22:31:16.285995 676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:16.286216 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 676
I20260811 22:31:16.286360 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:16.286743 19278 external_mini_cluster.cc:1428] Setting key f562d761e97959df34c11d01e080c8e6
I20260811 22:31:16.289580 819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:31:16 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:16.296859 819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Bootstrap starting.
I20260811 22:31:16.299201 819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:16.300341 819 log.cc:824] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:16.302119 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.288220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.18.211.190:43271)
0811 22:31:16.288647 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:16.288667 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:16.288719 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:31:16.289511 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:16.289532 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:16.289943 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:16.290254 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:16.291217 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.292261 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.293503 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.293978 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.294099 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.297091 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:16.297096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:16.297109 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:16.297139 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:16.298991 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.299543 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.299548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.299553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.299686 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.299994 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.299999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.300000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.300449 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:16.300651 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:16.301043 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 22:31:16.301389 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":147,"threads_started":1}
I20260811 22:31:16.302596 819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: No bootstrap required, opened a new log
I20260811 22:31:16.305975 819 raft_consensus.cc:348] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43271 } }
I20260811 22:31:16.306265 819 raft_consensus.cc:374] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:16.306348 819 raft_consensus.cc:729] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d19957c8e0874d0088e9b1c4ef5d828a, State: Initialized, Role: FOLLOWER
I20260811 22:31:16.306869 819 consensus_queue.cc:260] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43271 } }
I20260811 22:31:16.307081 819 raft_consensus.cc:388] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:16.307159 819 raft_consensus.cc:482] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:16.307278 819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:16.308386 819 raft_consensus.cc:504] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43271 } }
I20260811 22:31:16.308720 819 leader_election.cc:302] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d19957c8e0874d0088e9b1c4ef5d828a; no voters:
I20260811 22:31:16.309070 819 leader_election.cc:288] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:16.309324 830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:16.309602 830 raft_consensus.cc:686] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [term 1 LEADER]: Becoming Leader. State: Replica: d19957c8e0874d0088e9b1c4ef5d828a, State: Running, Role: LEADER
I20260811 22:31:16.309975 830 consensus_queue.cc:237] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43271 } }
I20260811 22:31:16.310722 819 sys_catalog.cc:564] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:16.311925 837 sys_catalog.cc:455] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d19957c8e0874d0088e9b1c4ef5d828a. Latest consensus state: current_term: 1 leader_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43271 } } }
I20260811 22:31:16.312067 837 sys_catalog.cc:458] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:16.312394 833 sys_catalog.cc:455] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19957c8e0874d0088e9b1c4ef5d828a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43271 } } }
I20260811 22:31:16.312493 848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:16.312512 833 sys_catalog.cc:458] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:16.316020 848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:16.323278 848 catalog_manager.cc:1347] Generated new cluster ID: 33ad5227a5bc4055bf4cce99a3082159
I20260811 22:31:16.323380 848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:16.360860 848 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:16.362298 848 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:16.387406 848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Generated new TSK 0
I20260811 22:31:16.388378 848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab.
Entry for principal oryx/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab.
I20260811 22:31:16.425206 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43271
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab
--principal=oryx/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:16.544817 864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:16.545302 864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:16.545406 864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:16.549605 864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:16.549669 864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:16.549708 864 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:31:16.549782 864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:16.555279 864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab
--principal=oryx/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:16.556363 864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:16.557654 864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:16.567554 875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:16.567543 874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:16.567546 878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:16.568020 864 server_base.cc:1034] running on GCE node
I20260811 22:31:16.568439 864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:16.569092 864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:16.570302 864 hybrid_clock.cc:648] HybridClock initialized: now 1786487476570279 us; error 31 us; skew 500 ppm
Aug 11 22:31:16 dist-test-slave-8hhm krb5kdc[654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:16.573529 864 init.cc:377] Logged in from keytab as oryx/127.18.211.129@KRBTEST.COM (short username oryx)
I20260811 22:31:16.574607 864 webserver.cc:460] Webserver started at http://127.18.211.129:41783/ using document root <none> and password file <none>
I20260811 22:31:16.575172 864 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:16.575233 864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:16.575412 864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:16.577601 864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "ee4c1181e7654a55ae778728257fef6c"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "8d0f3e88c6717d4842aca6c5fc93dca0"
server_key_iv: "4a3e798566dfa08d07108e503eaaf6f4"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.578161 864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "ee4c1181e7654a55ae778728257fef6c"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "8d0f3e88c6717d4842aca6c5fc93dca0"
server_key_iv: "4a3e798566dfa08d07108e503eaaf6f4"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.582093 864 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:31:16.584738 887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:16.586053 864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:31:16.586160 864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "ee4c1181e7654a55ae778728257fef6c"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "8d0f3e88c6717d4842aca6c5fc93dca0"
server_key_iv: "4a3e798566dfa08d07108e503eaaf6f4"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.586252 864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:16.600639 864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:16.604193 864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:16.604359 864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:16.605928 864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:16.605983 864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:16.606055 864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:16.606115 864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:31:16 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:16.617146 864 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:40595
I20260811 22:31:16.617334 1090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:40595 every 8 connection(s)
I20260811 22:31:16.618418 864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:16.621063 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.606713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:47147 (local address 127.18.211.190:43271)
0811 22:31:16.606876 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:16.606881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:16.607818 (+ 937us) server_negotiation.cc:407] Connection header received
0811 22:31:16.608926 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:16.608930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:16.608999 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:16.609066 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:16.611055 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.612029 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.613908 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.614142 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.614258 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.617436 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:16.617440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:16.617442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:16.617470 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:16.619096 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.619649 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.619652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.619653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.619746 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.620054 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.620056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.620057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.620239 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:16.620318 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:16.620834 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 22:31:16.620941 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:31:16.621912 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.607164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43271 (local address 127.18.211.129:47147)
0811 22:31:16.607624 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:16.607675 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:16.608649 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:16.609588 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:16.609597 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:16.610052 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:16.610861 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.610881 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.612190 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.612193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.613733 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.613745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.614312 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.614317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.614513 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.615173 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:16.615207 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:16.617229 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:16.619243 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.619249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.619264 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.619537 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.619832 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.619835 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.619836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.619964 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.620363 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:16.620369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:16.620611 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 22:31:16.620771 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:31:16.621030 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":127,"threads_started":1}
I20260811 22:31:16.623363 1091 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43271
I20260811 22:31:16.623548 1091 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:16.624181 1091 heartbeater.cc:511] Master 127.18.211.190:43271 requested a full tablet report, sending...
I20260811 22:31:16.624233 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 864
I20260811 22:31:16.624347 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:16.624797 19278 external_mini_cluster.cc:1428] Setting key a72514a2ec5b576268868cefd6b9f68a
I20260811 22:31:16.626060 726 ts_manager.cc:194] Registered new tserver with Master: ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595)
I20260811 22:31:16.627617 726 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:47147
I20260811 22:31:16.633807 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.625590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:37607 (local address 127.18.211.190:43271)
0811 22:31:16.626150 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:16.626154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:16.626168 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:16.626203 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:16.626206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:16.626244 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:16.626329 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:16.626868 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.627672 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.628803 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.629165 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.629361 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.630579 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:16.630582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:16.630584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:16.630687 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:16.632062 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.632554 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.632562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.632563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.632635 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.632930 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.632934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.632936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.633119 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:16.633274 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:16.633535 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 22:31:16.633660 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":491}
I20260811 22:31:16.633821 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.625527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43271 (local address 127.18.211.129:37607)
0811 22:31:16.625795 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:16.625817 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:16.625888 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:16.626369 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:16.626372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:16.626603 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:16.626737 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.626744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.627839 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.627842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.628664 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.628675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.629371 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.629375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.629464 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.630045 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:16.630059 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:16.630416 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:16.632181 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.632185 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.632187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.632450 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.632740 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.632743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.632745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.632836 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.633248 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:16.633251 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:16.633349 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 22:31:16.633467 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:31:16.633609 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for oryx/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab.
Entry for principal oryx/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab.
I20260811 22:31:16.656975 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43271
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab
--principal=oryx/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:16.776242 1106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:16.776546 1106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:16.776633 1106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:16.780748 1106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:16.780843 1106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:16.780918 1106 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:31:16.780998 1106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:16.786413 1106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab
--principal=oryx/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:16.787519 1106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:16.788604 1106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:16.798724 1120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:16.798703 1115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:16.798740 1113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:16.798990 1106 server_base.cc:1034] running on GCE node
I20260811 22:31:16.799432 1106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:16.800122 1106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:16.801414 1106 hybrid_clock.cc:648] HybridClock initialized: now 1786487476801391 us; error 30 us; skew 500 ppm
Aug 11 22:31:16 dist-test-slave-8hhm krb5kdc[654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:16.804468 1106 init.cc:377] Logged in from keytab as oryx/127.18.211.130@KRBTEST.COM (short username oryx)
I20260811 22:31:16.805395 1106 webserver.cc:460] Webserver started at http://127.18.211.130:34479/ using document root <none> and password file <none>
I20260811 22:31:16.805964 1106 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:16.806058 1106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:16.806243 1106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:16.808377 1106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "551cd9d03b6448c8a0b165a69eb3369b"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "012667e945feaba9838ba7258497378e"
server_key_iv: "05583a4f41a8aa9167c918f317f6a45f"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.808923 1106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "551cd9d03b6448c8a0b165a69eb3369b"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "012667e945feaba9838ba7258497378e"
server_key_iv: "05583a4f41a8aa9167c918f317f6a45f"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.813143 1106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:31:16.815835 1128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:16.817394 1106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:16.817508 1106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "551cd9d03b6448c8a0b165a69eb3369b"
format_stamp: "Formatted at 2026-08-11 22:31:16 on dist-test-slave-8hhm"
server_key: "012667e945feaba9838ba7258497378e"
server_key_iv: "05583a4f41a8aa9167c918f317f6a45f"
server_key_version: "encryptionkey@0"
I20260811 22:31:16.817608 1106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:16.828830 1106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:16.831996 1106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:16.832151 1106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:16.833657 1106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:16.833721 1106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:16.833762 1106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:16.833822 1106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:16.844271 1106 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:45279
Aug 11 22:31:16 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:16.844345 1325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:45279 every 8 connection(s)
I20260811 22:31:16.845399 1106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:16.848734 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.834524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:49759 (local address 127.18.211.190:43271)
0811 22:31:16.834699 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:16.834707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:16.835596 (+ 889us) server_negotiation.cc:407] Connection header received
0811 22:31:16.836778 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:16.836783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:16.836853 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:16.836920 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:16.838969 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.839844 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.841643 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.841884 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.842006 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.845046 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:16.845051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:16.845054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:16.845087 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:16.846953 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.847423 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.847426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.847427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.847505 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.847770 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.847772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.847773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.847950 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:16.848028 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:16.848489 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 22:31:16.848611 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:31:16.849696 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.834974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43271 (local address 127.18.211.130:49759)
0811 22:31:16.835423 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:16.835474 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:16.836482 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:16.837483 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:16.837492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:16.838034 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:16.838801 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.838824 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.839991 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.839996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.841513 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.841524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.842060 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.842065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.842295 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.843015 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:16.843043 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:16.844808 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:16.847059 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.847064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.847074 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.847333 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.847566 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.847568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.847569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.847693 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.848044 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:16.848049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:16.848279 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 22:31:16.848437 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 22:31:16.848759 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":123,"threads_started":1}
I20260811 22:31:16.851104 1326 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43271
I20260811 22:31:16.851284 1326 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:16.851837 1326 heartbeater.cc:511] Master 127.18.211.190:43271 requested a full tablet report, sending...
I20260811 22:31:16.852960 727 ts_manager.cc:194] Registered new tserver with Master: 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130:45279)
I20260811 22:31:16.853844 727 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:49759
I20260811 22:31:16.855293 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 1106
I20260811 22:31:16.855413 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:16.855767 19278 external_mini_cluster.cc:1428] Setting key 2b0c4dc36fd48183a9a18d0faebd1da4
I20260811 22:31:16.861459 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.853267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43271 (local address 127.18.211.130:52499)
0811 22:31:16.853604 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:16.853617 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:16.853721 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:16.853958 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:16.853961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:16.854192 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:16.854320 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.854327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.855335 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.855342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.856177 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:16.856193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.856710 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.856714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:16.856806 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.857385 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:16.857400 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:16.857882 (+ 482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:16.859721 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.859726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.859731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.860011 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.860300 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:16.860304 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:16.860309 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:16.860413 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:16.860842 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:16.860847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:16.860968 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 22:31:16.861097 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 22:31:16.861293 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
I20260811 22:31:16.861589 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:16.853358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:52499 (local address 127.18.211.190:43271)
0811 22:31:16.853491 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:16.853494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:16.853689 (+ 195us) server_negotiation.cc:407] Connection header received
0811 22:31:16.853802 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:16.853805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:16.853842 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:16.853909 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:16.854449 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.855182 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.856305 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:16.856587 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:16.856671 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:16.858073 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:16.858076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:16.858078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:16.858101 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:16.859600 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.860125 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.860127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.860128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.860208 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:16.860512 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:16.860515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:16.860516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:16.860744 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:16.860828 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:16.861180 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 22:31:16.861360 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for oryx/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab.
Entry for principal oryx/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab.
I20260811 22:31:16.889647 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43271
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab
--principal=oryx/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:17.009936 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:17.010248 1336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:17.010329 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:17.014578 1336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:17.014644 1336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:17.014712 1336 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:31:17.014801 1336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:17.020306 1336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab
--principal=oryx/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.211.190:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:17.021526 1336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:17.022629 1336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:17.032740 1346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:17.032735 1351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:17.032732 1344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:17.033027 1336 server_base.cc:1034] running on GCE node
I20260811 22:31:17.033633 1336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:17.034375 1336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:17.035567 1336 hybrid_clock.cc:648] HybridClock initialized: now 1786487477035547 us; error 31 us; skew 500 ppm
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487477, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:17.038602 1336 init.cc:377] Logged in from keytab as oryx/127.18.211.131@KRBTEST.COM (short username oryx)
I20260811 22:31:17.039525 1336 webserver.cc:460] Webserver started at http://127.18.211.131:38477/ using document root <none> and password file <none>
I20260811 22:31:17.040122 1336 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:17.040225 1336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:17.040475 1336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:17.042634 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "b17bd3d358a0413d94fc8d7fd487421a"
format_stamp: "Formatted at 2026-08-11 22:31:17 on dist-test-slave-8hhm"
server_key: "612c01aff030ffaef093902485b504ca"
server_key_iv: "3ab43f769ba265ba9fd967a72d22b46c"
server_key_version: "encryptionkey@0"
I20260811 22:31:17.043242 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "b17bd3d358a0413d94fc8d7fd487421a"
format_stamp: "Formatted at 2026-08-11 22:31:17 on dist-test-slave-8hhm"
server_key: "612c01aff030ffaef093902485b504ca"
server_key_iv: "3ab43f769ba265ba9fd967a72d22b46c"
server_key_version: "encryptionkey@0"
I20260811 22:31:17.047230 1336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:17.049925 1363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:17.051388 1336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:17.051570 1336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "b17bd3d358a0413d94fc8d7fd487421a"
format_stamp: "Formatted at 2026-08-11 22:31:17 on dist-test-slave-8hhm"
server_key: "612c01aff030ffaef093902485b504ca"
server_key_iv: "3ab43f769ba265ba9fd967a72d22b46c"
server_key_version: "encryptionkey@0"
I20260811 22:31:17.051729 1336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:17.069006 1336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:17.073195 1336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:17.073495 1336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:17.075073 1336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:17.075165 1336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:17.075239 1336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:17.075289 1336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487477, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:17.086944 1336 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:43723
I20260811 22:31:17.087240 1561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:43723 every 8 connection(s)
I20260811 22:31:17.088155 1336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:17.088953 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 1336
I20260811 22:31:17.089095 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:17.089514 19278 external_mini_cluster.cc:1428] Setting key 4b062b85da1ad584dab9ba0eaf9f2ee0
I20260811 22:31:17.090380 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.076263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:44131 (local address 127.18.211.190:43271)
0811 22:31:17.076438 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.076443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.077403 (+ 960us) server_negotiation.cc:407] Connection header received
0811 22:31:17.078833 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.078836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.078907 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.079008 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:17.080382 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.080930 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.082274 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.082517 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.082640 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.085908 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:17.085914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:17.085922 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:17.085950 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:17.087831 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.088444 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.088448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.088450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.088560 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.089068 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.089073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.089076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.089365 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:17.089479 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.090080 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.090218 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:31:17.091377 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.076653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43271 (local address 127.18.211.131:44131)
0811 22:31:17.077169 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.077280 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.078563 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.079221 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.079231 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.079597 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:17.080225 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.080240 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.081075 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.081079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.082137 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.082148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.082700 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.082704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.082893 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.083575 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:17.083595 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:17.085716 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:17.087970 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:17.087975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:17.087986 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:17.088327 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:17.088715 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:17.088719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:17.088720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:17.088922 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:17.089489 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:17.089495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:17.089806 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.090016 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.090383 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":113,"threads_started":1}
I20260811 22:31:17.092504 1566 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43271
I20260811 22:31:17.092643 1566 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:17.093257 1566 heartbeater.cc:511] Master 127.18.211.190:43271 requested a full tablet report, sending...
I20260811 22:31:17.094332 726 ts_manager.cc:194] Registered new tserver with Master: b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.095108 726 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:44131
I20260811 22:31:17.101868 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.094966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:60711 (local address 127.18.211.190:43271)
0811 22:31:17.095088 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.095092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.095267 (+ 175us) server_negotiation.cc:407] Connection header received
0811 22:31:17.095507 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.095509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.095546 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.095606 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:17.096121 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.096817 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.097674 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.097878 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.097957 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.098933 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:17.098936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:17.098938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:17.098959 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:17.100383 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.100769 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.100772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.100773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.100853 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.101098 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.101100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.101102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.101362 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:17.101445 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.101670 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.101762 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:31:17.101908 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.094881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43271 (local address 127.18.211.131:60711)
0811 22:31:17.095179 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.095192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.095304 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.095646 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.095649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.095870 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:17.095994 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.096000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.096945 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.096947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.097571 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.097580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.097974 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.097977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.098047 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.098494 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:17.098505 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:17.098812 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:17.100474 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:17.100476 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:17.100477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:17.100652 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:17.100938 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:17.100940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:17.100942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:17.101014 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:17.101446 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:17.101448 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:17.101522 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.101606 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.101723 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
I20260811 22:31:17.103519 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:31:17.130497 826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:17.124092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.18.211.190:43271)
0811 22:31:17.124292 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.124296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.124308 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:31:17.124352 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.124355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.124396 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.124465 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:17.125264 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.125793 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.126665 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.126863 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.126935 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.130363 (+ 3428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52954: BlockingRecv error: recv got EOF from 127.0.0.1:52954 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:17.140322 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.131880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.18.211.190:43271)
0811 22:31:17.132013 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.132017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.132118 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:31:17.132267 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.132270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.132318 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.132404 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:17.132983 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.133583 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.134378 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.134593 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.134703 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.137061 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:17.137065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:17.137067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:17.137091 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:17.138623 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.139076 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.139079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.139080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.139154 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.139434 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.139438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.139439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.139617 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:17.139768 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.140017 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.140133 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:17.143373 726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:31:17.146562 726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:17.161935 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.155807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:45279 (local address 127.0.0.1:55558)
0811 22:31:17.156925 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.156938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.157027 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.157806 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.157812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.157851 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:17.158177 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.158186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.159285 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.159288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.160474 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.160484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.161366 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.161370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.161490 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.161534 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.161650 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.161763 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":310,"spinlock_wait_cycles":7552,"thread_start_us":104,"threads_started":1}
I20260811 22:31:17.161962 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.156510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.18.211.130:45279)
0811 22:31:17.157061 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.157071 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.157092 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:31:17.157169 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.157174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.157464 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.157618 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:17.158321 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.159147 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.160603 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.161224 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.161381 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.161524 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.161705 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.161801 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":125,"threads_started":1}
I20260811 22:31:17.162940 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.155649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:43723 (local address 127.0.0.1:42970)
0811 22:31:17.156577 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.156634 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.156782 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.157806 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.157812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.157850 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:17.158182 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.158188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.159584 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.159588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.161043 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.161056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.162539 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.162542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.162617 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.162634 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.162738 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.162807 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":195,"spinlock_wait_cycles":5376,"thread_start_us":78,"threads_started":1}
I20260811 22:31:17.162990 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.155782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.18.211.131:43723)
0811 22:31:17.157317 (+ 1535us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.157327 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.157345 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:31:17.157407 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.157410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.157610 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.157774 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:17.158308 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.159438 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.161257 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.162252 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.162395 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.162486 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.162770 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.162857 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1384,"thread_start_us":125,"threads_started":1}
I20260811 22:31:17.163672 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.156542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.18.211.129:40595)
0811 22:31:17.157024 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.157031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.157502 (+ 471us) server_negotiation.cc:407] Connection header received
0811 22:31:17.157619 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.157627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.157800 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.157931 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:17.158257 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.159462 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.161199 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.162043 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.162166 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.162314 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.163457 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.163552 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":75,"threads_started":1}
I20260811 22:31:17.163909 1606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.156105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:40595 (local address 127.0.0.1:41890)
0811 22:31:17.157383 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.157398 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.157497 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.157903 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.157906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.157915 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:17.158136 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.158143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.159584 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.159587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.161046 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.161056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.163060 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.163063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.163148 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.163183 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.163257 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.163318 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"mutex_wait_us":213,"thread_start_us":125,"threads_started":1}
I20260811 22:31:17.165499 1211 tablet_service.cc:1467] Processing CreateTablet for tablet f8b09335079b47c8974084e32861e69b (DEFAULT_TABLE table=test-table [id=e0c21e08f3df4cbba2c4494bb8587340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.165978 1452 tablet_service.cc:1467] Processing CreateTablet for tablet f8b09335079b47c8974084e32861e69b (DEFAULT_TABLE table=test-table [id=e0c21e08f3df4cbba2c4494bb8587340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.166667 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b09335079b47c8974084e32861e69b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.167038 1452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b09335079b47c8974084e32861e69b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.166878 963 tablet_service.cc:1467] Processing CreateTablet for tablet f8b09335079b47c8974084e32861e69b (DEFAULT_TABLE table=test-table [id=e0c21e08f3df4cbba2c4494bb8587340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.167889 963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b09335079b47c8974084e32861e69b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.173794 1617 tablet_bootstrap.cc:492] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Bootstrap starting.
I20260811 22:31:17.175323 1619 tablet_bootstrap.cc:492] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Bootstrap starting.
I20260811 22:31:17.176403 1617 tablet_bootstrap.cc:654] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.177392 1617 log.cc:824] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:17.177614 1619 tablet_bootstrap.cc:654] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.177611 1618 tablet_bootstrap.cc:492] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Bootstrap starting.
I20260811 22:31:17.178406 1619 log.cc:824] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:17.179679 1617 tablet_bootstrap.cc:492] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: No bootstrap required, opened a new log
I20260811 22:31:17.179775 1618 tablet_bootstrap.cc:654] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.179888 1617 ts_tablet_manager.cc:1397] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 22:31:17.180460 1619 tablet_bootstrap.cc:492] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: No bootstrap required, opened a new log
I20260811 22:31:17.180486 1618 log.cc:824] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:17.180641 1619 ts_tablet_manager.cc:1397] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:31:17.182159 1618 tablet_bootstrap.cc:492] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: No bootstrap required, opened a new log
I20260811 22:31:17.182363 1618 ts_tablet_manager.cc:1397] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:31:17.184064 1617 raft_consensus.cc:348] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.184372 1617 raft_consensus.cc:374] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.184427 1617 raft_consensus.cc:729] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.184598 1619 raft_consensus.cc:348] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.184923 1619 raft_consensus.cc:374] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.184919 1617 consensus_queue.cc:260] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.185000 1619 raft_consensus.cc:729] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17bd3d358a0413d94fc8d7fd487421a, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.185484 1619 consensus_queue.cc:260] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.186010 1617 ts_tablet_manager.cc:1428] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260811 22:31:17.186276 1566 heartbeater.cc:503] Master 127.18.211.190:43271 was elected leader, sending a full tablet report...
I20260811 22:31:17.186374 1326 heartbeater.cc:503] Master 127.18.211.190:43271 was elected leader, sending a full tablet report...
I20260811 22:31:17.186565 1618 raft_consensus.cc:348] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.186817 1618 raft_consensus.cc:374] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.186868 1618 raft_consensus.cc:729] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee4c1181e7654a55ae778728257fef6c, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.187263 1618 consensus_queue.cc:260] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.187388 1619 ts_tablet_manager.cc:1428] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260811 22:31:17.187845 1091 heartbeater.cc:503] Master 127.18.211.190:43271 was elected leader, sending a full tablet report...
I20260811 22:31:17.188154 1618 ts_tablet_manager.cc:1428] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:31:17.296591 1626 raft_consensus.cc:482] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:17.296823 1626 raft_consensus.cc:504] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.298151 1626 leader_election.cc:288] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723), ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595)
I20260811 22:31:17.302567 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.298548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:59853 (local address 127.18.211.131:43723)
0811 22:31:17.298720 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.298724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.298743 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:31:17.298849 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.298853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.298909 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.299001 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:17.299341 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.300151 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.301429 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.302018 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.302125 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.302168 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.302406 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.302465 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:31:17.302587 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.298396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:43723 (local address 127.18.211.130:59853)
0811 22:31:17.298598 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.298615 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.298730 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.299044 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.299047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.299070 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:17.299211 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.299221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.300283 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.300286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.301312 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.301322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.302150 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.302153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.302274 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.302303 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.302381 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.302452 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 22:31:17.303294 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.298633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:51211 (local address 127.18.211.129:40595)
0811 22:31:17.298758 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.298765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.299017 (+ 252us) server_negotiation.cc:407] Connection header received
0811 22:31:17.299134 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.299137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.299198 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.299288 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:17.300056 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.300753 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.302041 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.302663 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.302783 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.302849 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.303139 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.303204 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:17.303323 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.298466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:40595 (local address 127.18.211.130:51211)
0811 22:31:17.298897 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.298911 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.299026 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.299339 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.299342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.299360 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:17.299930 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.299940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.300876 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.300879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.301905 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.301916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.302864 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.302868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.302969 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.303020 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.303106 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.303171 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":158,"threads_started":1}
I20260811 22:31:17.303352 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b09335079b47c8974084e32861e69b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a" is_pre_election: true
I20260811 22:31:17.303716 1486 raft_consensus.cc:2454] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 0.
I20260811 22:31:17.303936 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b09335079b47c8974084e32861e69b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c" is_pre_election: true
I20260811 22:31:17.304283 1002 raft_consensus.cc:2454] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 0.
I20260811 22:31:17.304311 1132 leader_election.cc:302] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, b17bd3d358a0413d94fc8d7fd487421a; no voters:
I20260811 22:31:17.304652 1626 raft_consensus.cc:2781] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:17.304772 1626 raft_consensus.cc:482] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:17.304839 1626 raft_consensus.cc:3037] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.306161 1626 raft_consensus.cc:504] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.306545 1626 leader_election.cc:288] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Requested vote from peers b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723), ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595)
I20260811 22:31:17.306928 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b09335079b47c8974084e32861e69b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a"
I20260811 22:31:17.306942 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b09335079b47c8974084e32861e69b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c"
I20260811 22:31:17.307067 1486 raft_consensus.cc:3037] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.307103 1002 raft_consensus.cc:3037] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.308403 1002 raft_consensus.cc:2454] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.308403 1486 raft_consensus.cc:2454] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.308763 1129 leader_election.cc:302] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, b17bd3d358a0413d94fc8d7fd487421a, ee4c1181e7654a55ae778728257fef6c; no voters:
I20260811 22:31:17.308983 1626 raft_consensus.cc:2781] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:17.309195 1626 raft_consensus.cc:686] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Becoming Leader. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Running, Role: LEADER
I20260811 22:31:17.309594 1626 consensus_queue.cc:237] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } }
I20260811 22:31:17.312863 727 catalog_manager.cc:5495] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b reported cstate change: term changed from 0 to 1, leader changed from <none> to 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130). New cstate: current_term: 1 leader_uuid: "551cd9d03b6448c8a0b165a69eb3369b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } health_report { overall_health: UNKNOWN } } }
W20260811 22:31:17.339807 1568 tablet.cc:2369] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:31:17.346683 1328 tablet.cc:2369] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:31:17.351652 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.348243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.18.211.130:45279)
0811 22:31:17.348375 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.348379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.348391 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:31:17.348465 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.348467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.348523 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.348615 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:17.348832 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.349348 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.350151 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.350358 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.350429 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.350656 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.351218 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.351306 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.351404 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.351456 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:31:17.357652 1486 raft_consensus.cc:1261] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.357671 1002 raft_consensus.cc:1261] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.358412 1647 consensus_queue.cc:1035] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.358665 1626 consensus_queue.cc:1035] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.365945 1654 mvcc.cc:205] Tried to move back new op lower bound from 7317452707249430528 to 7317452707062734848. Current Snapshot: MvccSnapshot[applied={T|T < 7317452707249430528}]
I20260811 22:31:17.367564 1659 mvcc.cc:205] Tried to move back new op lower bound from 7317452707249430528 to 7317452707062734848. Current Snapshot: MvccSnapshot[applied={T|T < 7317452707249430528}]
I20260811 22:31:17.368253 1657 mvcc.cc:205] Tried to move back new op lower bound from 7317452707249430528 to 7317452707062734848. Current Snapshot: MvccSnapshot[applied={T|T < 7317452707249430528}]
I20260811 22:31:17.374935 726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:17.375133 726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:17.377748 726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Sending DeleteTablet for 3 replicas of tablet f8b09335079b47c8974084e32861e69b
I20260811 22:31:17.378561 963 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b09335079b47c8974084e32861e69b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:41890
I20260811 22:31:17.378561 1211 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b09335079b47c8974084e32861e69b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55558
I20260811 22:31:17.379122 1669 tablet_replica.cc:330] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: stopping tablet replica
I20260811 22:31:17.379118 1452 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b09335079b47c8974084e32861e69b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:42970
I20260811 22:31:17.379576 1669 raft_consensus.cc:2229] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:17.379629 1670 tablet_replica.cc:330] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: stopping tablet replica
I20260811 22:31:17.379961 1670 raft_consensus.cc:2229] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:17.380074 727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:31:17.380301 1670 raft_consensus.cc:2258] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.380198 1669 raft_consensus.cc:2258] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:31:17.380520 727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:17.380635 1672 tablet_replica.cc:330] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: stopping tablet replica
I20260811 22:31:17.381068 1672 raft_consensus.cc:2229] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:17.381404 1672 raft_consensus.cc:2258] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.382915 1669 ts_tablet_manager.cc:1905] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.385670 963 tablet_service.cc:1467] Processing CreateTablet for tablet a004a79a83814f5cae0d3a673bd4a9a7 (DEFAULT_TABLE table=test-table [id=ebb0a797f63a4877a4ca7ff0a0044b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.385753 1211 tablet_service.cc:1467] Processing CreateTablet for tablet a004a79a83814f5cae0d3a673bd4a9a7 (DEFAULT_TABLE table=test-table [id=ebb0a797f63a4877a4ca7ff0a0044b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.385972 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a004a79a83814f5cae0d3a673bd4a9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.385972 963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a004a79a83814f5cae0d3a673bd4a9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.388346 1617 tablet_bootstrap.cc:492] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: Bootstrap starting.
I20260811 22:31:17.388564 1452 tablet_service.cc:1467] Processing CreateTablet for tablet a004a79a83814f5cae0d3a673bd4a9a7 (DEFAULT_TABLE table=test-table [id=ebb0a797f63a4877a4ca7ff0a0044b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.388854 1452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a004a79a83814f5cae0d3a673bd4a9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.389394 1617 tablet_bootstrap.cc:654] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.389420 1618 tablet_bootstrap.cc:492] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Bootstrap starting.
I20260811 22:31:17.390412 1618 tablet_bootstrap.cc:654] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.391673 1619 tablet_bootstrap.cc:492] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: Bootstrap starting.
I20260811 22:31:17.392704 1670 ts_tablet_manager.cc:1905] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.392755 1619 tablet_bootstrap.cc:654] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.393039 1617 tablet_bootstrap.cc:492] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: No bootstrap required, opened a new log
I20260811 22:31:17.393127 1617 ts_tablet_manager.cc:1397] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 22:31:17.393498 1672 ts_tablet_manager.cc:1905] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.393658 1617 raft_consensus.cc:348] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.393810 1617 raft_consensus.cc:374] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.393847 1617 raft_consensus.cc:729] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.394001 1617 consensus_queue.cc:260] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.394311 1617 ts_tablet_manager.cc:1428] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:31:17.394364 1618 tablet_bootstrap.cc:492] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: No bootstrap required, opened a new log
I20260811 22:31:17.394440 1618 ts_tablet_manager.cc:1397] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:31:17.394905 1618 raft_consensus.cc:348] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.395022 1618 raft_consensus.cc:374] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.395056 1618 raft_consensus.cc:729] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee4c1181e7654a55ae778728257fef6c, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.395190 1618 consensus_queue.cc:260] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.395467 1618 ts_tablet_manager.cc:1428] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:17.396024 1672 ts_tablet_manager.cc:1918] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:17.396116 1672 log.cc:1196] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/f8b09335079b47c8974084e32861e69b
I20260811 22:31:17.396463 1672 ts_tablet_manager.cc:1939] T f8b09335079b47c8974084e32861e69b P b17bd3d358a0413d94fc8d7fd487421a: Deleting consensus metadata
I20260811 22:31:17.396667 1619 tablet_bootstrap.cc:492] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: No bootstrap required, opened a new log
I20260811 22:31:17.396754 1619 ts_tablet_manager.cc:1397] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260811 22:31:17.397204 1619 raft_consensus.cc:348] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.397369 1619 raft_consensus.cc:374] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.397415 1619 raft_consensus.cc:729] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17bd3d358a0413d94fc8d7fd487421a, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.397534 1619 consensus_queue.cc:260] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.397828 1619 ts_tablet_manager.cc:1428] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:31:17.398587 711 catalog_manager.cc:4841] TS b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723): tablet f8b09335079b47c8974084e32861e69b (table test-table [id=e0c21e08f3df4cbba2c4494bb8587340]) successfully deleted
I20260811 22:31:17.400137 1670 ts_tablet_manager.cc:1918] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:17.400256 1670 log.cc:1196] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/f8b09335079b47c8974084e32861e69b
I20260811 22:31:17.400602 1670 ts_tablet_manager.cc:1939] T f8b09335079b47c8974084e32861e69b P ee4c1181e7654a55ae778728257fef6c: Deleting consensus metadata
I20260811 22:31:17.401870 703 catalog_manager.cc:4841] TS ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595): tablet f8b09335079b47c8974084e32861e69b (table test-table [id=e0c21e08f3df4cbba2c4494bb8587340]) successfully deleted
I20260811 22:31:17.402305 1669 ts_tablet_manager.cc:1918] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:17.402438 1669 log.cc:1196] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/f8b09335079b47c8974084e32861e69b
I20260811 22:31:17.402743 1669 ts_tablet_manager.cc:1939] T f8b09335079b47c8974084e32861e69b P 551cd9d03b6448c8a0b165a69eb3369b: Deleting consensus metadata
I20260811 22:31:17.403849 710 catalog_manager.cc:4841] TS 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130:45279): tablet f8b09335079b47c8974084e32861e69b (table test-table [id=e0c21e08f3df4cbba2c4494bb8587340]) successfully deleted
I20260811 22:31:17.408826 1626 raft_consensus.cc:482] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:17.408950 1626 raft_consensus.cc:504] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.409456 1626 leader_election.cc:288] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.409881 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004a79a83814f5cae0d3a673bd4a9a7" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a" is_pre_election: true
I20260811 22:31:17.409880 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004a79a83814f5cae0d3a673bd4a9a7" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c" is_pre_election: true
I20260811 22:31:17.410012 1486 raft_consensus.cc:2454] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 0.
I20260811 22:31:17.410038 1002 raft_consensus.cc:2454] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 0.
I20260811 22:31:17.410313 1132 leader_election.cc:302] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, b17bd3d358a0413d94fc8d7fd487421a; no voters:
I20260811 22:31:17.410490 1626 raft_consensus.cc:2781] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:17.410547 1626 raft_consensus.cc:482] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:17.410588 1626 raft_consensus.cc:3037] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.411893 1626 raft_consensus.cc:504] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.412271 1626 leader_election.cc:288] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Requested vote from peers ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.412638 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004a79a83814f5cae0d3a673bd4a9a7" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a"
I20260811 22:31:17.412652 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004a79a83814f5cae0d3a673bd4a9a7" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c"
I20260811 22:31:17.412744 1486 raft_consensus.cc:3037] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.412768 1002 raft_consensus.cc:3037] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.413801 1486 raft_consensus.cc:2454] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.413964 1002 raft_consensus.cc:2454] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.414176 1132 leader_election.cc:302] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, b17bd3d358a0413d94fc8d7fd487421a; no voters:
I20260811 22:31:17.414362 1626 raft_consensus.cc:2781] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:17.414463 1626 raft_consensus.cc:686] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Becoming Leader. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Running, Role: LEADER
I20260811 22:31:17.414610 1626 consensus_queue.cc:237] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.416275 726 catalog_manager.cc:5495] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b reported cstate change: term changed from 0 to 1, leader changed from <none> to 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130). New cstate: current_term: 1 leader_uuid: "551cd9d03b6448c8a0b165a69eb3369b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487476, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.190@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:17.460364 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.451305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.18.211.190:43271)
0811 22:31:17.451450 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.451455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.451631 (+ 176us) server_negotiation.cc:407] Connection header received
0811 22:31:17.451826 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.451828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.451868 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.451951 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:17.452747 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.453331 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.454120 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.454338 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.454447 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.457041 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:17.457045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:17.457048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:17.457080 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:17.458535 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.459025 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.459030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.459032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.459112 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.459393 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.459397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.459399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.459593 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:17.459711 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.460012 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.460177 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":50}
I20260811 22:31:17.460376 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.451215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43271 (local address 127.0.0.1:52964)
0811 22:31:17.451537 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.451549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.451696 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.451970 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.451973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.452518 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:17.452619 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.452626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.453439 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.453444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.453998 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.454006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.454454 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.454458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.454537 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.455206 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:17.455226 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:17.456878 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:17.458653 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:17.458664 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:17.458672 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:17.458920 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:17.459211 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:17.459214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:17.459216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:17.459303 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:17.459705 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:17.459712 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:17.459839 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.459964 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.460247 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":111,"thread_start_us":110,"threads_started":1}
I20260811 22:31:17.464850 726 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:52964:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:31:17.465713 726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:17.471256 963 tablet_service.cc:1467] Processing CreateTablet for tablet 49b2a943549a48b593b0190e2988d8d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a59d7d583b2c4011a2395bd3ef6814e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:31:17.471360 1211 tablet_service.cc:1467] Processing CreateTablet for tablet 49b2a943549a48b593b0190e2988d8d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a59d7d583b2c4011a2395bd3ef6814e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:31:17.471720 963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b2a943549a48b593b0190e2988d8d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.471776 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b2a943549a48b593b0190e2988d8d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.474359 1618 tablet_bootstrap.cc:492] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: Bootstrap starting.
I20260811 22:31:17.474596 1617 tablet_bootstrap.cc:492] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: Bootstrap starting.
I20260811 22:31:17.474839 1452 tablet_service.cc:1467] Processing CreateTablet for tablet 49b2a943549a48b593b0190e2988d8d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a59d7d583b2c4011a2395bd3ef6814e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:31:17.475243 1452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b2a943549a48b593b0190e2988d8d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.475435 1618 tablet_bootstrap.cc:654] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.475464 1617 tablet_bootstrap.cc:654] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.477309 1617 tablet_bootstrap.cc:492] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: No bootstrap required, opened a new log
I20260811 22:31:17.477411 1617 ts_tablet_manager.cc:1397] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 22:31:17.477898 1617 raft_consensus.cc:348] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.478018 1617 raft_consensus.cc:374] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.478051 1617 raft_consensus.cc:729] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.478183 1617 consensus_queue.cc:260] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.478292 1619 tablet_bootstrap.cc:492] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: Bootstrap starting.
I20260811 22:31:17.478474 1617 ts_tablet_manager.cc:1428] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:17.478731 1626 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.478870 1626 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:31:17.479346 1619 tablet_bootstrap.cc:654] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.479803 1618 tablet_bootstrap.cc:492] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: No bootstrap required, opened a new log
I20260811 22:31:17.479895 1618 ts_tablet_manager.cc:1397] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 22:31:17.480384 1618 raft_consensus.cc:348] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.480474 1618 raft_consensus.cc:374] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.480495 1618 raft_consensus.cc:729] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee4c1181e7654a55ae778728257fef6c, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.480647 1618 consensus_queue.cc:260] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.480952 1618 ts_tablet_manager.cc:1428] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:31:17.480896 1638 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.481029 1638 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:31:17.482101 1619 tablet_bootstrap.cc:492] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: No bootstrap required, opened a new log
I20260811 22:31:17.482213 1619 ts_tablet_manager.cc:1397] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 22:31:17.482654 1619 raft_consensus.cc:348] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.482795 1619 raft_consensus.cc:374] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.482836 1619 raft_consensus.cc:729] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17bd3d358a0413d94fc8d7fd487421a, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.483035 1619 consensus_queue.cc:260] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.483345 1619 ts_tablet_manager.cc:1428] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:31:17.483384 1629 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.483556 1629 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:31:17.551244 1638 raft_consensus.cc:482] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:17.551398 1638 raft_consensus.cc:504] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.552639 1638 leader_election.cc:288] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130:45279), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.557013 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.552981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:37411 (local address 127.18.211.130:45279)
0811 22:31:17.553167 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.553171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.553469 (+ 298us) server_negotiation.cc:407] Connection header received
0811 22:31:17.553633 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.553638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.553698 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.553777 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:17.554113 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.554688 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.555977 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.556407 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.556503 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.556576 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.556789 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.556887 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:31:17.557016 1707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.552866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:45279 (local address 127.18.211.129:37411)
0811 22:31:17.553327 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.553340 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.553498 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.553815 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.553818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.553840 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:17.553988 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.553996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.554814 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.554820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.555855 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.555865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.556525 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.556532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.556642 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.556665 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.556758 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.556817 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":72,"thread_start_us":160,"threads_started":1}
I20260811 22:31:17.557122 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.553086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:39001 (local address 127.18.211.131:43723)
0811 22:31:17.553256 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.553260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.553677 (+ 417us) server_negotiation.cc:407] Connection header received
0811 22:31:17.553817 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.553820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.553864 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.553944 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:17.554260 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.554726 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.555847 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.556491 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.556631 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.556681 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.556918 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.556988 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:31:17.557497 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.552987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:43723 (local address 127.18.211.129:39001)
0811 22:31:17.553570 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.553586 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.553703 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.553973 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.553975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.553991 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:17.554132 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.554139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.554872 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.554875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.555726 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.555735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.556633 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.556637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.556774 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.556802 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.556879 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.556943 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260811 22:31:17.557633 1248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b2a943549a48b593b0190e2988d8d7" candidate_uuid: "ee4c1181e7654a55ae778728257fef6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551cd9d03b6448c8a0b165a69eb3369b" is_pre_election: true
I20260811 22:31:17.557901 1248 raft_consensus.cc:2454] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee4c1181e7654a55ae778728257fef6c in term 0.
I20260811 22:31:17.558059 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b2a943549a48b593b0190e2988d8d7" candidate_uuid: "ee4c1181e7654a55ae778728257fef6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a" is_pre_election: true
I20260811 22:31:17.558210 1486 raft_consensus.cc:2454] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee4c1181e7654a55ae778728257fef6c in term 0.
I20260811 22:31:17.558418 892 leader_election.cc:302] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, ee4c1181e7654a55ae778728257fef6c; no voters:
I20260811 22:31:17.558789 1638 raft_consensus.cc:2781] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:17.558876 1638 raft_consensus.cc:482] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:17.558933 1638 raft_consensus.cc:3037] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.560011 1638 raft_consensus.cc:504] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.560325 1638 leader_election.cc:288] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [CANDIDATE]: Term 1 election: Requested vote from peers 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130:45279), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.560714 1248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b2a943549a48b593b0190e2988d8d7" candidate_uuid: "ee4c1181e7654a55ae778728257fef6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551cd9d03b6448c8a0b165a69eb3369b"
I20260811 22:31:17.560724 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b2a943549a48b593b0190e2988d8d7" candidate_uuid: "ee4c1181e7654a55ae778728257fef6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a"
I20260811 22:31:17.560855 1248 raft_consensus.cc:3037] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.560874 1486 raft_consensus.cc:3037] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.561889 1486 raft_consensus.cc:2454] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee4c1181e7654a55ae778728257fef6c in term 1.
I20260811 22:31:17.561919 1248 raft_consensus.cc:2454] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee4c1181e7654a55ae778728257fef6c in term 1.
I20260811 22:31:17.562199 894 leader_election.cc:302] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17bd3d358a0413d94fc8d7fd487421a, ee4c1181e7654a55ae778728257fef6c; no voters:
I20260811 22:31:17.562407 1638 raft_consensus.cc:2781] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:17.562628 1638 raft_consensus.cc:686] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [term 1 LEADER]: Becoming Leader. State: Replica: ee4c1181e7654a55ae778728257fef6c, State: Running, Role: LEADER
I20260811 22:31:17.563100 1638 consensus_queue.cc:237] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.564474 1708 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: TxnStatusTablet state changed. Reason: New leader ee4c1181e7654a55ae778728257fef6c. Latest consensus state: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.564615 1708 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:31:17.565255 1714 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:31:17.565414 1714 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:31:17.565886 726 catalog_manager.cc:5495] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c reported cstate change: term changed from 0 to 1, leader changed from <none> to ee4c1181e7654a55ae778728257fef6c (127.18.211.129). New cstate: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } health_report { overall_health: UNKNOWN } } }
I20260811 22:31:17.588048 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.584129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:40595 (local address 127.0.0.1:41898)
0811 22:31:17.584287 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.584305 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.584417 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.584780 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.584783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.584791 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:31:17.584909 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.584915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.585782 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.585787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.586407 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.586416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.586882 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.586886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.586982 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.587030 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.587743 (+ 713us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.587761 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.587836 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.587929 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 22:31:17.588048 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.584278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.18.211.129:40595)
0811 22:31:17.584424 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.584429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.584445 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:17.584594 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.584597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.584652 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.584768 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:17.585024 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.585670 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.586534 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.586758 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.586870 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.587197 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.587634 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.587774 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.587846 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.587900 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:31:17.591857 1486 raft_consensus.cc:1261] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Refusing update from remote peer ee4c1181e7654a55ae778728257fef6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.591952 1248 raft_consensus.cc:1261] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Refusing update from remote peer ee4c1181e7654a55ae778728257fef6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.592425 1708 consensus_queue.cc:1035] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.592727 1638 consensus_queue.cc:1035] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.594677 1629 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: TxnStatusTablet state changed. Reason: New leader ee4c1181e7654a55ae778728257fef6c. Latest consensus state: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.594810 1629 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:31:17.594741 1626 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: TxnStatusTablet state changed. Reason: New leader ee4c1181e7654a55ae778728257fef6c. Latest consensus state: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.594859 1626 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:31:17.597327 1638 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.597455 1638 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:31:17.598856 1626 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.598973 1626 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P 551cd9d03b6448c8a0b165a69eb3369b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:31:17.599982 1638 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.600096 1638 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P ee4c1181e7654a55ae778728257fef6c: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:31:17.602710 1629 tablet_replica.cc:439] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee4c1181e7654a55ae778728257fef6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } } }
I20260811 22:31:17.602829 1629 tablet_replica.cc:442] T 49b2a943549a48b593b0190e2988d8d7 P b17bd3d358a0413d94fc8d7fd487421a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:31:17.620338 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.613228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:50505 (local address 127.18.211.129:40595)
0811 22:31:17.613361 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.613367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.616234 (+ 2867us) server_negotiation.cc:407] Connection header received
0811 22:31:17.616387 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.616390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.616437 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.616496 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:17.616789 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.617704 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.618959 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.619398 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.619511 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.619855 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.619948 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.620023 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.620153 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.620214 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:31:17.620397 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.613060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:40595 (local address 127.18.211.130:50505)
0811 22:31:17.616101 (+ 3041us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.616138 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.616272 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.616531 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.616534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.616542 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:31:17.616684 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.616691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.617826 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.617830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.618816 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.618828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.619530 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.619534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.619644 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.619702 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.620041 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.620048 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.620115 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.620231 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2882,"thread_start_us":159,"threads_started":1}
W20260811 22:31:17.620532 1092 tablet.cc:2369] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:31:17.629698 1486 raft_consensus.cc:1261] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.629693 1002 raft_consensus.cc:1261] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.630133 1626 consensus_queue.cc:1035] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.630383 1647 consensus_queue.cc:1035] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.660847 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.655980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:48191 (local address 127.18.211.130:45279)
0811 22:31:17.656088 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.656092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.656193 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:31:17.656380 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.656384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.656435 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.656527 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:17.656888 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.657739 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.658820 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.659241 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.659344 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.660210 (+ 866us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.660320 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.660396 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.660632 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.660703 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:31:17.660871 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.655491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:45279 (local address 127.18.211.129:48191)
0811 22:31:17.656096 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:17.656114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:17.656243 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:17.656579 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:17.656582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:17.656590 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:31:17.656768 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.656775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.657868 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.657870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.658688 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:17.658699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.659846 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.659850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:17.659951 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.660013 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.660529 (+ 516us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.660536 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:31:17.660614 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:31:17.660698 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":169,"threads_started":1}
I20260811 22:31:17.690121 726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:17.690338 726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:17.693161 726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Sending DeleteTablet for 3 replicas of tablet a004a79a83814f5cae0d3a673bd4a9a7
I20260811 22:31:17.693845 963 tablet_service.cc:1514] Processing DeleteTablet for tablet a004a79a83814f5cae0d3a673bd4a9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:41890
I20260811 22:31:17.693970 1211 tablet_service.cc:1514] Processing DeleteTablet for tablet a004a79a83814f5cae0d3a673bd4a9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55558
I20260811 22:31:17.694012 1452 tablet_service.cc:1514] Processing DeleteTablet for tablet a004a79a83814f5cae0d3a673bd4a9a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:42970
I20260811 22:31:17.694204 1669 tablet_replica.cc:330] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: stopping tablet replica
I20260811 22:31:17.694213 1670 tablet_replica.cc:330] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: stopping tablet replica
I20260811 22:31:17.694326 1670 raft_consensus.cc:2229] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:17.694336 1669 raft_consensus.cc:2229] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:17.694442 1670 raft_consensus.cc:2258] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.694532 1669 raft_consensus.cc:2258] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.694639 1672 tablet_replica.cc:330] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: stopping tablet replica
I20260811 22:31:17.694795 1672 raft_consensus.cc:2229] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:17.694923 1672 raft_consensus.cc:2258] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.695626 1669 ts_tablet_manager.cc:1905] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.696054 1672 ts_tablet_manager.cc:1905] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.696054 1670 ts_tablet_manager.cc:1905] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.699447 1669 ts_tablet_manager.cc:1918] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:31:17.699563 1669 log.cc:1196] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/a004a79a83814f5cae0d3a673bd4a9a7
I20260811 22:31:17.699636 1670 ts_tablet_manager.cc:1918] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:31:17.699888 1670 log.cc:1196] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/a004a79a83814f5cae0d3a673bd4a9a7
I20260811 22:31:17.700309 1669 ts_tablet_manager.cc:1939] T a004a79a83814f5cae0d3a673bd4a9a7 P 551cd9d03b6448c8a0b165a69eb3369b: Deleting consensus metadata
I20260811 22:31:17.700307 1672 ts_tablet_manager.cc:1918] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:31:17.700470 1672 log.cc:1196] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/a004a79a83814f5cae0d3a673bd4a9a7
I20260811 22:31:17.700479 1670 ts_tablet_manager.cc:1939] T a004a79a83814f5cae0d3a673bd4a9a7 P ee4c1181e7654a55ae778728257fef6c: Deleting consensus metadata
I20260811 22:31:17.700814 1672 ts_tablet_manager.cc:1939] T a004a79a83814f5cae0d3a673bd4a9a7 P b17bd3d358a0413d94fc8d7fd487421a: Deleting consensus metadata
I20260811 22:31:17.701094 710 catalog_manager.cc:4841] TS 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130:45279): tablet a004a79a83814f5cae0d3a673bd4a9a7 (table test-table [id=ebb0a797f63a4877a4ca7ff0a0044b7e]) successfully deleted
I20260811 22:31:17.701264 703 catalog_manager.cc:4841] TS ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595): tablet a004a79a83814f5cae0d3a673bd4a9a7 (table test-table [id=ebb0a797f63a4877a4ca7ff0a0044b7e]) successfully deleted
I20260811 22:31:17.702323 711 catalog_manager.cc:4841] TS b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723): tablet a004a79a83814f5cae0d3a673bd4a9a7 (table test-table [id=ebb0a797f63a4877a4ca7ff0a0044b7e]) successfully deleted
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:31:17.742142 826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:17.732587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.18.211.190:43271)
0811 22:31:17.732881 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.732889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.733057 (+ 168us) server_negotiation.cc:407] Connection header received
0811 22:31:17.733323 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.733328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.733443 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.733670 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:17.734408 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.735295 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.736329 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.736780 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.736907 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.741993 (+ 5086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52980: BlockingRecv error: recv got EOF from 127.0.0.1:52980 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 11 22:31:17 dist-test-slave-8hhm krb5kdc[654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM
I20260811 22:31:17.758093 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.744564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.18.211.190:43271)
0811 22:31:17.744763 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.744767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.744889 (+ 122us) server_negotiation.cc:407] Connection header received
0811 22:31:17.745051 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.745053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.745095 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.745165 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:17.746109 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.746608 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.748729 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.748993 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.749114 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.753338 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:17.753357 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:17.753367 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:17.753427 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:17.755837 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.756297 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.756301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.756303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.756395 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:17.756752 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:17.756755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:17.756757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:17.756966 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:17.757153 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.757681 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.757808 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:31:17.762220 726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:31:17.762718 726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:17.768451 963 tablet_service.cc:1467] Processing CreateTablet for tablet 66a7c2bff0d648cdb934492e9a0e6f3b (DEFAULT_TABLE table=test-table [id=6b5828dd625242a1b0ee234a618ca2eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.768661 1211 tablet_service.cc:1467] Processing CreateTablet for tablet 66a7c2bff0d648cdb934492e9a0e6f3b (DEFAULT_TABLE table=test-table [id=6b5828dd625242a1b0ee234a618ca2eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.768918 963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a7c2bff0d648cdb934492e9a0e6f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.769063 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a7c2bff0d648cdb934492e9a0e6f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.771579 1617 tablet_bootstrap.cc:492] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: Bootstrap starting.
I20260811 22:31:17.772123 1452 tablet_service.cc:1467] Processing CreateTablet for tablet 66a7c2bff0d648cdb934492e9a0e6f3b (DEFAULT_TABLE table=test-table [id=6b5828dd625242a1b0ee234a618ca2eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.772399 1617 tablet_bootstrap.cc:654] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.772428 1452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a7c2bff0d648cdb934492e9a0e6f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.773758 1618 tablet_bootstrap.cc:492] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: Bootstrap starting.
I20260811 22:31:17.774215 1617 tablet_bootstrap.cc:492] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: No bootstrap required, opened a new log
I20260811 22:31:17.774362 1617 ts_tablet_manager.cc:1397] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 22:31:17.775068 1617 raft_consensus.cc:348] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.775295 1617 raft_consensus.cc:374] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.775331 1617 raft_consensus.cc:729] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.775348 1618 tablet_bootstrap.cc:654] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.775606 1617 consensus_queue.cc:260] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.775753 1626 raft_consensus.cc:482] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:17.775828 1626 raft_consensus.cc:504] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.776345 1626 leader_election.cc:288] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.776741 1618 tablet_bootstrap.cc:492] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: No bootstrap required, opened a new log
I20260811 22:31:17.776851 1618 ts_tablet_manager.cc:1397] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:17.776870 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7c2bff0d648cdb934492e9a0e6f3b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c" is_pre_election: true
I20260811 22:31:17.777287 1618 raft_consensus.cc:348] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.777323 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7c2bff0d648cdb934492e9a0e6f3b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a" is_pre_election: true
I20260811 22:31:17.777477 1618 raft_consensus.cc:374] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.777518 1618 raft_consensus.cc:729] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee4c1181e7654a55ae778728257fef6c, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.777717 1618 consensus_queue.cc:260] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.778080 1618 ts_tablet_manager.cc:1428] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:31:17.778092 1619 tablet_bootstrap.cc:492] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: Bootstrap starting.
W20260811 22:31:17.778460 1132 leader_election.cc:341] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723): Illegal state: must be running to vote when last-logged opid is not known
I20260811 22:31:17.779070 1002 raft_consensus.cc:2454] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 0.
I20260811 22:31:17.779174 1619 tablet_bootstrap.cc:654] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.779474 1129 leader_election.cc:302] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, ee4c1181e7654a55ae778728257fef6c; no voters: b17bd3d358a0413d94fc8d7fd487421a
I20260811 22:31:17.779749 1626 raft_consensus.cc:2781] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:17.779809 1626 raft_consensus.cc:482] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:17.779851 1626 raft_consensus.cc:3037] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.780988 1617 ts_tablet_manager.cc:1428] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260811 22:31:17.781080 1626 raft_consensus.cc:504] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.781464 1626 leader_election.cc:288] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Requested vote from peers ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.782199 1619 tablet_bootstrap.cc:492] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: No bootstrap required, opened a new log
I20260811 22:31:17.782321 1619 ts_tablet_manager.cc:1397] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 22:31:17.783056 1619 raft_consensus.cc:348] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.783125 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7c2bff0d648cdb934492e9a0e6f3b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a"
I20260811 22:31:17.783232 1619 raft_consensus.cc:374] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.783267 1619 raft_consensus.cc:729] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17bd3d358a0413d94fc8d7fd487421a, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.783423 1619 consensus_queue.cc:260] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.783479 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7c2bff0d648cdb934492e9a0e6f3b" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c"
I20260811 22:31:17.783615 1002 raft_consensus.cc:3037] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.783785 1619 ts_tablet_manager.cc:1428] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:31:17.785120 1486 raft_consensus.cc:3037] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.785550 1002 raft_consensus.cc:2454] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.785962 1129 leader_election.cc:302] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, ee4c1181e7654a55ae778728257fef6c; no voters:
I20260811 22:31:17.786268 1626 raft_consensus.cc:2781] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:17.786880 1626 raft_consensus.cc:686] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Becoming Leader. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Running, Role: LEADER
I20260811 22:31:17.787029 1626 consensus_queue.cc:237] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.787207 1486 raft_consensus.cc:2454] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.790550 727 catalog_manager.cc:5495] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b reported cstate change: term changed from 0 to 1, leader changed from <none> to 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130). New cstate: current_term: 1 leader_uuid: "551cd9d03b6448c8a0b165a69eb3369b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } health_report { overall_health: UNKNOWN } } }
I20260811 22:31:17.805135 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:17.801405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55564 (local address 127.18.211.130:45279)
0811 22:31:17.801620 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:17.801625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:17.801641 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:17.801749 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:17.801752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:17.801814 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:17.801899 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:17.802161 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.802839 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.803793 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:17.804171 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:17.804268 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:17.804651 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:17.804725 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:17.804781 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:17.804906 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 22:31:17.804990 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 22:31:17.808367 1486 raft_consensus.cc:1261] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.808482 1002 raft_consensus.cc:1261] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.809377 1730 consensus_queue.cc:1035] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.809631 1626 consensus_queue.cc:1035] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.819391 725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:17.819623 725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:17.821893 725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Sending DeleteTablet for 3 replicas of tablet 66a7c2bff0d648cdb934492e9a0e6f3b
I20260811 22:31:17.823218 1211 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a7c2bff0d648cdb934492e9a0e6f3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55558
I20260811 22:31:17.823326 1452 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a7c2bff0d648cdb934492e9a0e6f3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:42970
I20260811 22:31:17.823562 1672 tablet_replica.cc:330] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: stopping tablet replica
I20260811 22:31:17.823568 1669 tablet_replica.cc:330] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: stopping tablet replica
I20260811 22:31:17.823720 1672 raft_consensus.cc:2229] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:17.823719 1669 raft_consensus.cc:2229] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:17.823887 1672 raft_consensus.cc:2258] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.823939 1669 raft_consensus.cc:2258] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.824167 725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:31:17.824505 963 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a7c2bff0d648cdb934492e9a0e6f3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:17 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:41890
I20260811 22:31:17.824828 1670 tablet_replica.cc:330] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: stopping tablet replica
I20260811 22:31:17.824952 1670 raft_consensus.cc:2229] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 22:31:17.824956 725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:17.825098 1670 raft_consensus.cc:2258] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:17.825124 1669 ts_tablet_manager.cc:1905] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.826174 1670 ts_tablet_manager.cc:1905] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.826942 1672 ts_tablet_manager.cc:1905] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:17.828142 1669 ts_tablet_manager.cc:1918] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:17.828203 1669 log.cc:1196] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/66a7c2bff0d648cdb934492e9a0e6f3b
I20260811 22:31:17.828401 1669 ts_tablet_manager.cc:1939] T 66a7c2bff0d648cdb934492e9a0e6f3b P 551cd9d03b6448c8a0b165a69eb3369b: Deleting consensus metadata
I20260811 22:31:17.829094 710 catalog_manager.cc:4841] TS 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130:45279): tablet 66a7c2bff0d648cdb934492e9a0e6f3b (table test-table [id=6b5828dd625242a1b0ee234a618ca2eb]) successfully deleted
I20260811 22:31:17.829301 1670 ts_tablet_manager.cc:1918] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:17.829380 1670 log.cc:1196] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/66a7c2bff0d648cdb934492e9a0e6f3b
I20260811 22:31:17.829658 1670 ts_tablet_manager.cc:1939] T 66a7c2bff0d648cdb934492e9a0e6f3b P ee4c1181e7654a55ae778728257fef6c: Deleting consensus metadata
I20260811 22:31:17.829741 1672 ts_tablet_manager.cc:1918] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:17.829818 1672 log.cc:1196] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/66a7c2bff0d648cdb934492e9a0e6f3b
I20260811 22:31:17.830066 1672 ts_tablet_manager.cc:1939] T 66a7c2bff0d648cdb934492e9a0e6f3b P b17bd3d358a0413d94fc8d7fd487421a: Deleting consensus metadata
I20260811 22:31:17.830528 703 catalog_manager.cc:4841] TS ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595): tablet 66a7c2bff0d648cdb934492e9a0e6f3b (table test-table [id=6b5828dd625242a1b0ee234a618ca2eb]) successfully deleted
I20260811 22:31:17.831115 711 catalog_manager.cc:4841] TS b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723): tablet 66a7c2bff0d648cdb934492e9a0e6f3b (table test-table [id=6b5828dd625242a1b0ee234a618ca2eb]) successfully deleted
I20260811 22:31:17.833390 963 tablet_service.cc:1467] Processing CreateTablet for tablet 707154a284b2461f82e7e7af37bba085 (DEFAULT_TABLE table=test-table [id=24b14abb91d14e7581b1b9b3c9e5be41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.833506 1211 tablet_service.cc:1467] Processing CreateTablet for tablet 707154a284b2461f82e7e7af37bba085 (DEFAULT_TABLE table=test-table [id=24b14abb91d14e7581b1b9b3c9e5be41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.833637 1452 tablet_service.cc:1467] Processing CreateTablet for tablet 707154a284b2461f82e7e7af37bba085 (DEFAULT_TABLE table=test-table [id=24b14abb91d14e7581b1b9b3c9e5be41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:17.834139 963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707154a284b2461f82e7e7af37bba085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.834422 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707154a284b2461f82e7e7af37bba085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.834493 1452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707154a284b2461f82e7e7af37bba085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:17.836866 1618 tablet_bootstrap.cc:492] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c: Bootstrap starting.
I20260811 22:31:17.837443 1617 tablet_bootstrap.cc:492] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: Bootstrap starting.
I20260811 22:31:17.837901 1618 tablet_bootstrap.cc:654] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.838521 1617 tablet_bootstrap.cc:654] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.839322 1618 tablet_bootstrap.cc:492] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c: No bootstrap required, opened a new log
I20260811 22:31:17.839419 1618 ts_tablet_manager.cc:1397] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:17.839871 1619 tablet_bootstrap.cc:492] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: Bootstrap starting.
I20260811 22:31:17.839913 1617 tablet_bootstrap.cc:492] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: No bootstrap required, opened a new log
I20260811 22:31:17.840134 1617 ts_tablet_manager.cc:1397] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 22:31:17.840412 1618 raft_consensus.cc:348] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.840524 1618 raft_consensus.cc:374] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.840557 1618 raft_consensus.cc:729] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee4c1181e7654a55ae778728257fef6c, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.840698 1618 consensus_queue.cc:260] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.840731 1617 raft_consensus.cc:348] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.841084 1617 raft_consensus.cc:374] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.841128 1617 raft_consensus.cc:729] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.841233 1619 tablet_bootstrap.cc:654] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:17.841295 1617 consensus_queue.cc:260] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.841660 1617 ts_tablet_manager.cc:1428] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:31:17.842957 1619 tablet_bootstrap.cc:492] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: No bootstrap required, opened a new log
I20260811 22:31:17.843061 1619 ts_tablet_manager.cc:1397] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:17.843624 1618 ts_tablet_manager.cc:1428] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:31:17.843962 1619 raft_consensus.cc:348] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.844236 1619 raft_consensus.cc:374] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:17.844273 1619 raft_consensus.cc:729] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17bd3d358a0413d94fc8d7fd487421a, State: Initialized, Role: FOLLOWER
I20260811 22:31:17.844420 1619 consensus_queue.cc:260] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.844741 1619 ts_tablet_manager.cc:1428] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:31:17.932193 1626 raft_consensus.cc:482] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:17.932369 1626 raft_consensus.cc:504] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.932895 1626 leader_election.cc:288] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.933459 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707154a284b2461f82e7e7af37bba085" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a" is_pre_election: true
I20260811 22:31:17.933475 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707154a284b2461f82e7e7af37bba085" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c" is_pre_election: true
I20260811 22:31:17.933696 1486 raft_consensus.cc:2454] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 0.
I20260811 22:31:17.933692 1002 raft_consensus.cc:2454] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 0.
I20260811 22:31:17.934031 1129 leader_election.cc:302] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, ee4c1181e7654a55ae778728257fef6c; no voters:
I20260811 22:31:17.934231 1626 raft_consensus.cc:2781] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:17.934293 1626 raft_consensus.cc:482] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:17.934321 1626 raft_consensus.cc:3037] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.935511 1626 raft_consensus.cc:504] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.935950 1626 leader_election.cc:288] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Requested vote from peers ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595), b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723)
I20260811 22:31:17.936416 1002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707154a284b2461f82e7e7af37bba085" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4c1181e7654a55ae778728257fef6c"
I20260811 22:31:17.936440 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707154a284b2461f82e7e7af37bba085" candidate_uuid: "551cd9d03b6448c8a0b165a69eb3369b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bd3d358a0413d94fc8d7fd487421a"
I20260811 22:31:17.936617 1002 raft_consensus.cc:3037] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.936628 1486 raft_consensus.cc:3037] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:17.938444 1486 raft_consensus.cc:2454] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.938588 1002 raft_consensus.cc:2454] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551cd9d03b6448c8a0b165a69eb3369b in term 1.
I20260811 22:31:17.938951 1132 leader_election.cc:302] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551cd9d03b6448c8a0b165a69eb3369b, b17bd3d358a0413d94fc8d7fd487421a; no voters:
I20260811 22:31:17.939159 1626 raft_consensus.cc:2781] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:17.939347 1626 raft_consensus.cc:686] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Becoming Leader. State: Replica: 551cd9d03b6448c8a0b165a69eb3369b, State: Running, Role: LEADER
I20260811 22:31:17.939635 1626 consensus_queue.cc:237] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } }
I20260811 22:31:17.941843 727 catalog_manager.cc:5495] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b reported cstate change: term changed from 0 to 1, leader changed from <none> to 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130). New cstate: current_term: 1 leader_uuid: "551cd9d03b6448c8a0b165a69eb3369b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551cd9d03b6448c8a0b165a69eb3369b" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 } health_report { overall_health: UNKNOWN } } }
I20260811 22:31:17.985947 1486 raft_consensus.cc:1261] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.986009 1002 raft_consensus.cc:1261] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Refusing update from remote peer 551cd9d03b6448c8a0b165a69eb3369b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:17.986393 1647 consensus_queue.cc:1035] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17bd3d358a0413d94fc8d7fd487421a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:17.986588 1626 consensus_queue.cc:1035] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee4c1181e7654a55ae778728257fef6c" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:18.024561 727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:18.024737 727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:18.027238 727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d19957c8e0874d0088e9b1c4ef5d828a: Sending DeleteTablet for 3 replicas of tablet 707154a284b2461f82e7e7af37bba085
I20260811 22:31:18.028139 963 tablet_service.cc:1514] Processing DeleteTablet for tablet 707154a284b2461f82e7e7af37bba085 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:18 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:41890
I20260811 22:31:18.028250 1211 tablet_service.cc:1514] Processing DeleteTablet for tablet 707154a284b2461f82e7e7af37bba085 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:18 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55558
I20260811 22:31:18.028383 1452 tablet_service.cc:1514] Processing DeleteTablet for tablet 707154a284b2461f82e7e7af37bba085 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:18 UTC) from {username='oryx', principal='oryx/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:42970
I20260811 22:31:18.028594 1670 tablet_replica.cc:330] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c: stopping tablet replica
I20260811 22:31:18.028605 1672 tablet_replica.cc:330] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: stopping tablet replica
I20260811 22:31:18.028733 1670 raft_consensus.cc:2229] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:18.028760 1672 raft_consensus.cc:2229] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:18.028849 1670 raft_consensus.cc:2258] T 707154a284b2461f82e7e7af37bba085 P ee4c1181e7654a55ae778728257fef6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:18.028883 1672 raft_consensus.cc:2258] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:18.029049 1669 tablet_replica.cc:330] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: stopping tablet replica
I20260811 22:31:18.029168 1669 raft_consensus.cc:2229] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:18.029299 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 864
I20260811 22:31:18.029449 1669 raft_consensus.cc:2258] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:18.036386 1672 ts_tablet_manager.cc:1905] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:18.040766 1672 ts_tablet_manager.cc:1918] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:31:18.040928 1672 log.cc:1196] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/707154a284b2461f82e7e7af37bba085
I20260811 22:31:18.042068 1672 ts_tablet_manager.cc:1939] T 707154a284b2461f82e7e7af37bba085 P b17bd3d358a0413d94fc8d7fd487421a: Deleting consensus metadata
I20260811 22:31:18.046945 1669 ts_tablet_manager.cc:1905] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:18.049826 711 catalog_manager.cc:4841] TS b17bd3d358a0413d94fc8d7fd487421a (127.18.211.131:43723): tablet 707154a284b2461f82e7e7af37bba085 (table test-table [id=24b14abb91d14e7581b1b9b3c9e5be41]) successfully deleted
I20260811 22:31:18.051644 1669 ts_tablet_manager.cc:1918] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:31:18.051846 1669 log.cc:1196] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/707154a284b2461f82e7e7af37bba085
I20260811 22:31:18.052327 1669 ts_tablet_manager.cc:1939] T 707154a284b2461f82e7e7af37bba085 P 551cd9d03b6448c8a0b165a69eb3369b: Deleting consensus metadata
I20260811 22:31:18.053536 710 catalog_manager.cc:4841] TS 551cd9d03b6448c8a0b165a69eb3369b (127.18.211.130:45279): tablet 707154a284b2461f82e7e7af37bba085 (table test-table [id=24b14abb91d14e7581b1b9b3c9e5be41]) successfully deleted
W20260811 22:31:18.057797 703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:40595 (error 108)
I20260811 22:31:18.058204 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 1106
I20260811 22:31:18.074852 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:18.059170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:40595 (local address 127.0.0.1:41904)
0811 22:31:18.074441 (+ 15271us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:18.074634 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14965,"thread_start_us":2943,"threads_started":1}
W20260811 22:31:18.075223 703 catalog_manager.cc:4584] TS ee4c1181e7654a55ae778728257fef6c (127.18.211.129:40595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 707154a284b2461f82e7e7af37bba085: Network error: Client connection negotiation failed: client connection to 127.18.211.129:40595: connect: Connection refused (error 111)
I20260811 22:31:18.085310 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 1336
I20260811 22:31:18.108491 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 676
2026-08-11T22:31:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3497 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:31:18 dist-test-slave-8hhm krb5kdc[1794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:31:18 dist-test-slave-8hhm krb5kdc[1794](info): set up 2 sockets
Aug 11 22:31:18 dist-test-slave-8hhm krb5kdc[1794](info): commencing operation
krb5kdc: starting...
W20260811 22:31:19.466866 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.003s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:19 dist-test-slave-8hhm krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:31:19.600083 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:38975
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:38975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:19.719242 1820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:19.719591 1820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:19.719684 1820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:19.723708 1820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:19.723805 1820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:19.723846 1820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:19.723884 1820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:19.723917 1820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:19.729328 1820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:38975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:38975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:19.730477 1820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:19.731563 1820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:19.740486 1826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:19.740505 1829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:19.740486 1825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:19.740723 1820 server_base.cc:1034] running on GCE node
I20260811 22:31:19.741466 1820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:19.742702 1820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:19.743904 1820 hybrid_clock.cc:648] HybridClock initialized: now 1786487479743885 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T22:31:19Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1629 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:31:19 dist-test-slave-8hhm krb5kdc[1841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:31:19 dist-test-slave-8hhm krb5kdc[1841](info): set up 2 sockets
Aug 11 22:31:19 dist-test-slave-8hhm krb5kdc[1841](info): commencing operation
krb5kdc: starting...
W20260811 22:31:21.073537 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:21 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:31:21.210601 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45711
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:21.327535 1864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:21.327888 1864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:21.327966 1864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:21.332054 1864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:21.332158 1864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:21.332192 1864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:21.332233 1864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:21.332293 1864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:21.337687 1864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:21.338817 1864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:21.339887 1864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:21.348567 1874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:21.348531 1878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:21.348582 1875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:21.348711 1864 server_base.cc:1034] running on GCE node
I20260811 22:31:21.349530 1864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:21.350661 1864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:21.351842 1864 hybrid_clock.cc:648] HybridClock initialized: now 1786487481351814 us; error 40 us; skew 500 ppm
Aug 11 22:31:21 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487481, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:21.354727 1864 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:31:21.355641 1864 webserver.cc:460] Webserver started at http://127.18.211.190:41027/ using document root <none> and password file <none>
I20260811 22:31:21.356243 1864 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:21.356307 1864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:21.356549 1864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:21.358891 1864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "86e18776b41f4bf29b24882c0015eb2b"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "9d4910b02ef97c87cca929e612a93473"
server_key_iv: "7e2d22f807a0855600821c1f8442eca8"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.359503 1864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "86e18776b41f4bf29b24882c0015eb2b"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "9d4910b02ef97c87cca929e612a93473"
server_key_iv: "7e2d22f807a0855600821c1f8442eca8"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.363513 1864 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:21.365975 1889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:21.367234 1864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:31:21.367422 1864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "86e18776b41f4bf29b24882c0015eb2b"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "9d4910b02ef97c87cca929e612a93473"
server_key_iv: "7e2d22f807a0855600821c1f8442eca8"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.367547 1864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:21.418488 1864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:21.422859 1864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:21.423132 1864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:21.430619 1864 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45711
I20260811 22:31:21.430641 1985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45711 every 8 connection(s)
I20260811 22:31:21.431651 1864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:21.434612 1986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:21.438930 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 1864
I20260811 22:31:21.439116 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:21.439409 19278 external_mini_cluster.cc:1428] Setting key b7633a9a04d356ade68303cc38831e59
I20260811 22:31:21.440754 1986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Bootstrap starting.
I20260811 22:31:21.443643 1986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:21.444656 1986 log.cc:824] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:21.446946 1986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: No bootstrap required, opened a new log
Aug 11 22:31:21 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:21.451951 1986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:21.452289 1986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:21.452347 1986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e18776b41f4bf29b24882c0015eb2b, State: Initialized, Role: FOLLOWER
I20260811 22:31:21.452848 1986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:21.453032 1986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:21.453114 1986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:21.453270 1986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:21.454586 1986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:21.455013 1986 leader_election.cc:302] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e18776b41f4bf29b24882c0015eb2b; no voters:
I20260811 22:31:21.455404 1986 leader_election.cc:288] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:21.455541 1993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:21.455796 1993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 LEADER]: Becoming Leader. State: Replica: 86e18776b41f4bf29b24882c0015eb2b, State: Running, Role: LEADER
I20260811 22:31:21.456110 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:21.440894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.18.211.190:45711)
0811 22:31:21.441343 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:21.441363 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:21.441407 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:31:21.442288 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:21.442313 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:21.442749 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:21.443132 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:21.443968 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.445428 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.446359 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.447060 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.447220 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:21.450137 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:21.450148 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:21.450174 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:21.450218 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:21.452442 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:21.452980 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:21.452987 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:21.452994 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:21.453159 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:21.453516 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:21.453523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:21.453527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:21.454088 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:21.454304 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:21.454580 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:31:21.454999 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
I20260811 22:31:21.456213 1993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:21.456795 1986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:21.458067 1994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e18776b41f4bf29b24882c0015eb2b. Latest consensus state: current_term: 1 leader_uuid: "86e18776b41f4bf29b24882c0015eb2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } } }
I20260811 22:31:21.458290 1994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:21.458716 1997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86e18776b41f4bf29b24882c0015eb2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } } }
I20260811 22:31:21.458875 1997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:21.459263 2015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:21.462486 2015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:21.468088 2015 catalog_manager.cc:1347] Generated new cluster ID: 040a75ffc9a4431bb9ad5bd349af52c5
I20260811 22:31:21.468171 2015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:21.497529 2015 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:21.498602 2015 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:21.506312 2015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Generated new TSK 0
I20260811 22:31:21.507079 2015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:31:21.544413 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45711
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:21.659781 2037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:21.660085 2037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:21.660131 2037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:21.664271 2037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:21.664341 2037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:21.664434 2037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:21.669721 2037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:21.670905 2037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:21.672143 2037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:21.682969 2050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:21.682979 2049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:21.682979 2054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:21.683362 2037 server_base.cc:1034] running on GCE node
I20260811 22:31:21.683766 2037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:21.684459 2037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:21.685658 2037 hybrid_clock.cc:648] HybridClock initialized: now 1786487481685628 us; error 51 us; skew 500 ppm
Aug 11 22:31:21 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487481, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:21.688535 2037 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:31:21.689486 2037 webserver.cc:460] Webserver started at http://127.18.211.129:46873/ using document root <none> and password file <none>
I20260811 22:31:21.690061 2037 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:21.690157 2037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:21.690433 2037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:21.692534 2037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "e2e083c687e14c7b9e0f96f73cc11fd3"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "606170fba0d2023cea0c6f9bae091823"
server_key_iv: "9de0c02e9c3668e590f587649a767ac5"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.693137 2037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "e2e083c687e14c7b9e0f96f73cc11fd3"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "606170fba0d2023cea0c6f9bae091823"
server_key_iv: "9de0c02e9c3668e590f587649a767ac5"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.696867 2037 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:21.699371 2062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:21.700655 2037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:21.700798 2037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "e2e083c687e14c7b9e0f96f73cc11fd3"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "606170fba0d2023cea0c6f9bae091823"
server_key_iv: "9de0c02e9c3668e590f587649a767ac5"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.700917 2037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:21.718356 2037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:21.721555 2037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:21.721772 2037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:21.722393 2037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:21.723263 2037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:21.723335 2037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:21.723409 2037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:21.723461 2037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:21.733048 2037 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:37751
I20260811 22:31:21.733095 2273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:37751 every 8 connection(s)
I20260811 22:31:21.734030 2037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:21.742105 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 2037
I20260811 22:31:21.742288 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:21.742599 19278 external_mini_cluster.cc:1428] Setting key 4a4b5ad18af82816c02645b184233209
Aug 11 22:31:21 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487481, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:21.748876 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:21.735986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:38693 (local address 127.18.211.190:45711)
0811 22:31:21.736135 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:21.736139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:21.737102 (+ 963us) server_negotiation.cc:407] Connection header received
0811 22:31:21.738629 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:21.738632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:21.738692 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:21.738801 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:21.740054 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.740574 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.741734 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.741986 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.742101 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:21.744935 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:21.744939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:21.744941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:21.744970 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:21.746491 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:21.747146 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:21.747149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:21.747150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:21.747227 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:21.747634 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:21.747636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:21.747637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:21.747809 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:21.747889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:21.748482 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 22:31:21.748633 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:31:21.749581 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:21.736375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:38693)
0811 22:31:21.736837 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:21.736879 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:21.738373 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:21.738989 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:21.738996 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:21.739365 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:21.739881 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:21.739894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.740707 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.740711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:21.741602 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:21.741611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.742156 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.742160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:21.742444 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:21.743101 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:21.743118 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:21.744769 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:21.746628 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:21.746638 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:21.746657 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:21.747041 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:21.747321 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:21.747327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:21.747329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:21.747533 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:21.747910 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:21.747915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:21.748195 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 22:31:21.748388 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 22:31:21.748646 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260811 22:31:21.750952 2274 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45711
I20260811 22:31:21.751219 2274 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:21.751724 2274 heartbeater.cc:511] Master 127.18.211.190:45711 requested a full tablet report, sending...
I20260811 22:31:21.753496 1920 ts_manager.cc:194] Registered new tserver with Master: e2e083c687e14c7b9e0f96f73cc11fd3 (127.18.211.129:37751)
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:31:21.755086 1920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:38693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:31:21.774415 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45711
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:21.891311 2282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:21.891574 2282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:21.891655 2282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:21.895685 2282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:21.895745 2282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:21.895855 2282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:21.901274 2282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:21.902570 2282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:21.903744 2282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:21.913504 2297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:21.913545 2291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:21.913504 2292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:21.913843 2282 server_base.cc:1034] running on GCE node
I20260811 22:31:21.914296 2282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:21.914882 2282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:21.916101 2282 hybrid_clock.cc:648] HybridClock initialized: now 1786487481916090 us; error 29 us; skew 500 ppm
Aug 11 22:31:21 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487481, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:21.918874 2282 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:31:21.919807 2282 webserver.cc:460] Webserver started at http://127.18.211.130:32875/ using document root <none> and password file <none>
I20260811 22:31:21.920380 2282 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:21.920473 2282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:21.920715 2282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:21.922842 2282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "7a69595dfe3a4adfa84742effc8a2125"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "c97f363e4ed6f85f0d1cc51ecf9672aa"
server_key_iv: "e9b24a74077e99b68417e3e42f1fb34b"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.923494 2282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "7a69595dfe3a4adfa84742effc8a2125"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "c97f363e4ed6f85f0d1cc51ecf9672aa"
server_key_iv: "e9b24a74077e99b68417e3e42f1fb34b"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.927301 2282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:21.929780 2309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:21.930989 2282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:21.931176 2282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "7a69595dfe3a4adfa84742effc8a2125"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "c97f363e4ed6f85f0d1cc51ecf9672aa"
server_key_iv: "e9b24a74077e99b68417e3e42f1fb34b"
server_key_version: "encryptionkey@0"
I20260811 22:31:21.931293 2282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:21.969129 2282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:21.972523 2282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:21.972739 2282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:21.973453 2282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:21.974366 2282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:21.974442 2282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:21.974520 2282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:21.974565 2282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:21.983891 2282 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:40163
I20260811 22:31:21.983920 2502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:40163 every 8 connection(s)
I20260811 22:31:21.984848 2282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:21.992627 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 2282
I20260811 22:31:21.992740 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:21.993144 19278 external_mini_cluster.cc:1428] Setting key e3551c1464fcd2752736ef34e5bc5880
Aug 11 22:31:21 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487481, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:22.000196 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:21.986868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:39119 (local address 127.18.211.190:45711)
0811 22:31:21.987020 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:21.987024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:21.987890 (+ 866us) server_negotiation.cc:407] Connection header received
0811 22:31:21.989327 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:21.989330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:21.989394 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:21.989499 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:21.990717 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.991240 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.992450 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.992671 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.992806 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:21.995881 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:21.995888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:21.995893 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:21.995931 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:21.997610 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:21.998321 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:21.998323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:21.998325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:21.998405 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:21.998871 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:21.998874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:21.998875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:21.999090 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:21.999172 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:21.999939 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 22:31:22.000069 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:31:22.001689 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:21.987281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:39119)
0811 22:31:21.987736 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:21.987779 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:21.989043 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:21.989691 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:21.989698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:21.990062 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:21.990558 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:21.990570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.991374 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.991382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:21.992288 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:21.992299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:21.992898 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:21.992903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:21.993130 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:21.993937 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:21.993955 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:21.995691 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:21.997732 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:21.997759 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:21.997779 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:21.998205 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:21.998500 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:21.998506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:21.998512 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:21.998752 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:21.999220 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:21.999231 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:21.999616 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 22:31:21.999878 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 22:31:22.000298 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260811 22:31:22.003145 2504 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45711
I20260811 22:31:22.003456 2504 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:22.004179 2504 heartbeater.cc:511] Master 127.18.211.190:45711 requested a full tablet report, sending...
I20260811 22:31:22.005618 1920 ts_manager.cc:194] Registered new tserver with Master: 7a69595dfe3a4adfa84742effc8a2125 (127.18.211.130:40163)
I20260811 22:31:22.006352 1920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:39119
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:31:22.028442 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45711
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:22.146639 2513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:22.146929 2513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:22.147001 2513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:22.151063 2513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:22.151149 2513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:22.151238 2513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:22.156503 2513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:22.157780 2513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:22.158972 2513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:22.169037 2524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:22.169065 2523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:22.169065 2527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:22.169559 2513 server_base.cc:1034] running on GCE node
I20260811 22:31:22.169996 2513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:22.170650 2513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:22.171845 2513 hybrid_clock.cc:648] HybridClock initialized: now 1786487482171826 us; error 39 us; skew 500 ppm
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487482, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:22.174783 2513 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:31:22.175654 2513 webserver.cc:460] Webserver started at http://127.18.211.131:35799/ using document root <none> and password file <none>
I20260811 22:31:22.176189 2513 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:22.176249 2513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:22.176417 2513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:22.178535 2513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "d6751f3aff9a4262b61ad35a0c8ec352"
format_stamp: "Formatted at 2026-08-11 22:31:22 on dist-test-slave-8hhm"
server_key: "17c1ccd2c8b330b4f2f82393319fc3c6"
server_key_iv: "1354499a1e5e8af3e98149780525f0c7"
server_key_version: "encryptionkey@0"
I20260811 22:31:22.179064 2513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "d6751f3aff9a4262b61ad35a0c8ec352"
format_stamp: "Formatted at 2026-08-11 22:31:22 on dist-test-slave-8hhm"
server_key: "17c1ccd2c8b330b4f2f82393319fc3c6"
server_key_iv: "1354499a1e5e8af3e98149780525f0c7"
server_key_version: "encryptionkey@0"
I20260811 22:31:22.182922 2513 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:22.185374 2549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.186688 2513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:22.186929 2513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "d6751f3aff9a4262b61ad35a0c8ec352"
format_stamp: "Formatted at 2026-08-11 22:31:22 on dist-test-slave-8hhm"
server_key: "17c1ccd2c8b330b4f2f82393319fc3c6"
server_key_iv: "1354499a1e5e8af3e98149780525f0c7"
server_key_version: "encryptionkey@0"
I20260811 22:31:22.187043 2513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:22.211288 2513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:22.214373 2513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:22.214601 2513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:22.215260 2513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:22.216156 2513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:22.216205 2513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.216279 2513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:22.216316 2513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.225241 2513 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:36129
I20260811 22:31:22.225338 2761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:36129 every 8 connection(s)
I20260811 22:31:22.226272 2513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:22.226644 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 2513
I20260811 22:31:22.226799 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:22.227180 19278 external_mini_cluster.cc:1428] Setting key 3debe6f8e2991a9ed8d209b91bb5e9ec
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487482, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:22.240193 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:22.228280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:38023 (local address 127.18.211.190:45711)
0811 22:31:22.228418 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:22.228423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:22.229364 (+ 941us) server_negotiation.cc:407] Connection header received
0811 22:31:22.230770 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:22.230772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:22.230818 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:22.230887 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:22.232066 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.232580 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.233789 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.234007 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.234093 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.236556 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:22.236559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:22.236561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:22.236588 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:22.238309 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:22.238772 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:22.238775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:22.238776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:22.238853 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:22.239146 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:22.239148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:22.239149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:22.239324 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:22.239400 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:22.239897 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 22:31:22.240032 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:31:22.241062 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:22.228712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:38023)
0811 22:31:22.229192 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:22.229269 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:22.230572 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:22.231063 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:22.231071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:22.231432 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:22.231943 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.231957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.232695 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.232697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.233674 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.233692 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.234136 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.234139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.234277 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.234828 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:22.234844 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:22.236410 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:22.238407 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:22.238412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:22.238424 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:22.238675 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:22.238932 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:22.238934 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:22.238936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:22.239069 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:22.239410 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:22.239416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:22.239648 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 22:31:22.239815 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 22:31:22.240146 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":132,"threads_started":1}
I20260811 22:31:22.242240 2764 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45711
I20260811 22:31:22.242499 2764 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:22.243009 2764 heartbeater.cc:511] Master 127.18.211.190:45711 requested a full tablet report, sending...
I20260811 22:31:22.244017 1920 ts_manager.cc:194] Registered new tserver with Master: d6751f3aff9a4262b61ad35a0c8ec352 (127.18.211.131:36129)
I20260811 22:31:22.244596 1920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:38023
I20260811 22:31:22.253407 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab.
Entry for principal oryx/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab.
WARNING: no policy specified for oryx/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab.
Entry for principal oryx/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab.
WARNING: no policy specified for oryx/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab.
Entry for principal oryx/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab.
I20260811 22:31:22.335397 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 2037
I20260811 22:31:22.369349 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 2282
I20260811 22:31:22.384692 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 2513
I20260811 22:31:22.397051 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 1864
I20260811 22:31:22.408434 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45711
--webserver_interface=127.18.211.190
--webserver_port=41027
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:22.525520 2777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:22.525820 2777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:22.525859 2777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:22.529877 2777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:22.529942 2777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:22.529960 2777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:22.529978 2777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:22.529992 2777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:22.535305 2777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=41027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:22.536433 2777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:22.537549 2777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:22.546104 2786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:22.546058 2789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:22.546084 2792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:22.546535 2777 server_base.cc:1034] running on GCE node
I20260811 22:31:22.547108 2777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:22.548120 2777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:22.549299 2777 hybrid_clock.cc:648] HybridClock initialized: now 1786487482549285 us; error 25 us; skew 500 ppm
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487482, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:22.552038 2777 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:31:22.552943 2777 webserver.cc:460] Webserver started at http://127.18.211.190:41027/ using document root <none> and password file <none>
I20260811 22:31:22.553537 2777 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:22.553618 2777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:22.557384 2777 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 22:31:22.559437 2805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.560739 2777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:22.560839 2777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "86e18776b41f4bf29b24882c0015eb2b"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "9d4910b02ef97c87cca929e612a93473"
server_key_iv: "7e2d22f807a0855600821c1f8442eca8"
server_key_version: "encryptionkey@0"
I20260811 22:31:22.561506 2777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:22.589540 2777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:22.592775 2777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:22.592998 2777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:22.600729 2777 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45711
I20260811 22:31:22.600755 2887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45711 every 8 connection(s)
I20260811 22:31:22.601925 2777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:22.606094 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 2777
I20260811 22:31:22.606657 2889 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:31:22.606647 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:37751
--local_ip_for_outbound_sockets=127.18.211.129
--tserver_master_addrs=127.18.211.190:45711
--webserver_port=46873
--webserver_interface=127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:31:22.609180 2889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Bootstrap starting.
I20260811 22:31:22.626581 2889 log.cc:824] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:22.633800 2889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:31:22.634225 2889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Bootstrap complete.
I20260811 22:31:22.638741 2889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:22.639153 2889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e18776b41f4bf29b24882c0015eb2b, State: Initialized, Role: FOLLOWER
I20260811 22:31:22.639688 2889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:22.639809 2889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:22.639897 2889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:22.639998 2889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:31:22.642252 2889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:22.642704 2889 leader_election.cc:302] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e18776b41f4bf29b24882c0015eb2b; no voters:
I20260811 22:31:22.643059 2889 leader_election.cc:288] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:31:22.643224 2896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:31:22.643469 2896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [term 2 LEADER]: Becoming Leader. State: Replica: 86e18776b41f4bf29b24882c0015eb2b, State: Running, Role: LEADER
I20260811 22:31:22.643791 2896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } }
I20260811 22:31:22.644361 2889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:22.645267 2901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e18776b41f4bf29b24882c0015eb2b. Latest consensus state: current_term: 2 leader_uuid: "86e18776b41f4bf29b24882c0015eb2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } } }
I20260811 22:31:22.645447 2901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:22.646137 2900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "86e18776b41f4bf29b24882c0015eb2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e18776b41f4bf29b24882c0015eb2b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45711 } } }
I20260811 22:31:22.646256 2900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:22.646245 2912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:22.649695 2912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:22.650347 2912 catalog_manager.cc:1259] Loaded cluster ID: 040a75ffc9a4431bb9ad5bd349af52c5
I20260811 22:31:22.650414 2912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:22.652732 2912 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:22.653476 2912 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 86e18776b41f4bf29b24882c0015eb2b: Loaded TSK: 0
I20260811 22:31:22.654686 2912 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:31:22.726581 2894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:22.726852 2894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:22.726894 2894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:22.731025 2894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:22.731096 2894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:22.731154 2894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:22.736505 2894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.129.keytab
--principal=oryx/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:37751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=46873
--tserver_master_addrs=127.18.211.190:45711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:22.737699 2894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:22.738781 2894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:22.750754 2940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:22.750887 2894 server_base.cc:1034] running on GCE node
W20260811 22:31:22.750783 2939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:22.751266 2942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:22.751775 2894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:22.752429 2894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:22.753621 2894 hybrid_clock.cc:648] HybridClock initialized: now 1786487482753591 us; error 49 us; skew 500 ppm
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487482, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:22.756544 2894 init.cc:377] Logged in from keytab as oryx/127.18.211.129@KRBTEST.COM (short username oryx)
I20260811 22:31:22.757741 2894 webserver.cc:460] Webserver started at http://127.18.211.129:46873/ using document root <none> and password file <none>
I20260811 22:31:22.758343 2894 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:22.758424 2894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:22.762647 2894 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:31:22.765491 2953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.766932 2894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:31:22.767045 2894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "e2e083c687e14c7b9e0f96f73cc11fd3"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "606170fba0d2023cea0c6f9bae091823"
server_key_iv: "9de0c02e9c3668e590f587649a767ac5"
server_key_version: "encryptionkey@0"
I20260811 22:31:22.767613 2894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:22.784952 2894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:22.788211 2894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:22.788432 2894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:22.789077 2894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:22.789995 2894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:22.790045 2894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.790122 2894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:22.790153 2894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.800098 2894 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:37751
I20260811 22:31:22.800120 3154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:37751 every 8 connection(s)
I20260811 22:31:22.801198 2894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:22.805042 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 2894
I20260811 22:31:22.805629 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:40163
--local_ip_for_outbound_sockets=127.18.211.130
--tserver_master_addrs=127.18.211.190:45711
--webserver_port=32875
--webserver_interface=127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:31:22.816985 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:22.803465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:39707 (local address 127.18.211.190:45711)
0811 22:31:22.803999 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:22.804027 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:22.804237 (+ 210us) server_negotiation.cc:407] Connection header received
0811 22:31:22.806305 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:22.806317 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:22.806662 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:22.806958 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:22.808345 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.810059 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.811385 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.812278 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.812486 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.816269 (+ 3783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:39707: BlockingRecv error: recv got EOF from 127.18.211.129:39707 (error 108)
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":151,"threads_started":1}
I20260811 22:31:22.817150 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:22.803595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:39707)
0811 22:31:22.804075 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:22.804126 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:22.805852 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:22.807124 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:22.807136 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:22.807643 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:22.808142 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.808158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.810201 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.810204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.811247 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.811258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.812445 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.812447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.812579 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.813202 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:22.813262 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:22.816061 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":124,"threads_started":1}
W20260811 22:31:22.817790 3155 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:45711 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.211.190:45711: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:22.823544 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:22.818245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:38159)
0811 22:31:22.818410 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:22.818427 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:22.818512 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:22.818798 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:22.818801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:22.818964 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:22.819053 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.819058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.819784 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.819788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.820322 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.820330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.820791 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.820795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.820908 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.821456 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:22.821468 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:22.823383 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 22:31:22.823613 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:22.818376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:38159 (local address 127.18.211.190:45711)
0811 22:31:22.818494 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:22.818498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:22.818511 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:22.818638 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:22.818641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:22.818694 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:22.818782 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:22.819167 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.819675 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.820443 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.820664 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.820788 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.823502 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:38159: BlockingRecv error: recv got EOF from 127.18.211.129:38159 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:22.829633 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:22.824305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:35657)
0811 22:31:22.824461 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:22.824476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:22.824555 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:22.824798 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:22.824804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:22.824991 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:22.825088 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.825094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.825837 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.825841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.826708 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:22.826718 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.827119 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.827122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:22.827214 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.827686 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:22.827701 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:22.829497 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 22:31:22.829726 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:22.824387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:35657 (local address 127.18.211.190:45711)
0811 22:31:22.824496 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:22.824499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:22.824512 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:22.824643 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:22.824644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:22.824681 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:22.824744 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:22.825198 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.825722 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.826819 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:22.827015 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:22.827088 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:22.829616 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:35657: BlockingRecv error: recv got EOF from 127.18.211.129:35657 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 22:31:22.829865 3155 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:31:22.929275 3162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:22.929536 3162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:22.929579 3162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:22.933504 3162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:22.933566 3162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:22.933624 3162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:22.938966 3162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.130.keytab
--principal=oryx/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:40163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=32875
--tserver_master_addrs=127.18.211.190:45711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:22.940016 3162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:22.941007 3162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:22.950945 3180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:22.951124 3162 server_base.cc:1034] running on GCE node
W20260811 22:31:22.950978 3175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:22.950945 3178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:22.951728 3162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:22.952451 3162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:22.953634 3162 hybrid_clock.cc:648] HybridClock initialized: now 1786487482953617 us; error 30 us; skew 500 ppm
Aug 11 22:31:22 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487482, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:22.956329 3162 init.cc:377] Logged in from keytab as oryx/127.18.211.130@KRBTEST.COM (short username oryx)
I20260811 22:31:22.957286 3162 webserver.cc:460] Webserver started at http://127.18.211.130:32875/ using document root <none> and password file <none>
I20260811 22:31:22.957854 3162 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:22.958029 3162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:22.961910 3162 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:31:22.964337 3193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.965912 3162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:22.966060 3162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "7a69595dfe3a4adfa84742effc8a2125"
format_stamp: "Formatted at 2026-08-11 22:31:21 on dist-test-slave-8hhm"
server_key: "c97f363e4ed6f85f0d1cc51ecf9672aa"
server_key_iv: "e9b24a74077e99b68417e3e42f1fb34b"
server_key_version: "encryptionkey@0"
I20260811 22:31:22.966686 3162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:22.982133 3162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:22.985826 3162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:22.986052 3162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:22.986694 3162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:22.987578 3162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:22.987653 3162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.987740 3162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:22.987797 3162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:22.997345 3162 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:40163
I20260811 22:31:22.997355 3378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:40163 every 8 connection(s)
I20260811 22:31:22.998422 3162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:23.004452 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 3162
I20260811 22:31:23.004923 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:36129
--local_ip_for_outbound_sockets=127.18.211.131
--tserver_master_addrs=127.18.211.190:45711
--webserver_port=35799
--webserver_interface=127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:31:23.010396 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:23.000397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:51035 (local address 127.18.211.190:45711)
0811 22:31:23.000567 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.000572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.001469 (+ 897us) server_negotiation.cc:407] Connection header received
0811 22:31:23.002878 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.002882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.002950 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.003061 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.004301 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.004888 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.006177 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.006408 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.006542 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.010268 (+ 3726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:51035: BlockingRecv error: recv got EOF from 127.18.211.130:51035 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:31:23.011044 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.000808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:51035)
0811 22:31:23.001311 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:23.001356 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:23.002636 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:23.003247 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:23.003254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:23.003624 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:23.004140 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.004154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.005014 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.005018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.006037 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.006047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.006632 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.006639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.006829 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.007482 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:23.007499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:23.010120 (+ 2621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":118,"threads_started":1}
W20260811 22:31:23.011636 3379 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:45711 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.211.190:45711: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:23.017396 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.012163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:39569)
0811 22:31:23.012279 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:23.012295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:23.012375 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:23.012609 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:23.012612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:23.012802 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:23.012892 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.012898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.013678 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.013682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.014243 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.014252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.014695 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.014699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.014792 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.015302 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:23.015314 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:23.017253 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:31:23.017488 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:23.012273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:39569 (local address 127.18.211.190:45711)
0811 22:31:23.012396 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.012403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.012419 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:23.012460 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.012462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.012508 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.012592 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.013011 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.013569 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.014370 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.014583 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.014700 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.017369 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:39569: BlockingRecv error: recv got EOF from 127.18.211.130:39569 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:23.023334 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.018067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:45205)
0811 22:31:23.018211 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:23.018226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:23.018302 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:23.018527 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:23.018530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:23.018698 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:23.018790 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.018796 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.019463 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.019466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.020323 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.020334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.020787 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.020790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.020871 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.021394 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:23.021408 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:23.023209 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:31:23.023551 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:23.018131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:45205 (local address 127.18.211.190:45711)
0811 22:31:23.018254 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.018257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.018269 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:31:23.018388 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.018390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.018426 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.018487 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.018899 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.019370 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.020436 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.020652 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.020738 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.023325 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:45205: BlockingRecv error: recv got EOF from 127.18.211.130:45205 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:31:23.023695 3379 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:31:23.124779 3384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:23.125061 3384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:23.125141 3384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:23.129984 3384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:23.130045 3384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:23.130105 3384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:23.135377 3384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/krb5kdc/oryx_127.18.211.131.keytab
--principal=oryx/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:36129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=35799
--tserver_master_addrs=127.18.211.190:45711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:23.136443 3384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:23.137531 3384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:23.147758 3393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:23.147742 3403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:23.147739 3392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:23.148003 3384 server_base.cc:1034] running on GCE node
I20260811 22:31:23.148485 3384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:23.149134 3384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:23.150300 3384 hybrid_clock.cc:648] HybridClock initialized: now 1786487483150283 us; error 26 us; skew 500 ppm
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487483, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:23.153399 3384 init.cc:377] Logged in from keytab as oryx/127.18.211.131@KRBTEST.COM (short username oryx)
I20260811 22:31:23.154361 3384 webserver.cc:460] Webserver started at http://127.18.211.131:35799/ using document root <none> and password file <none>
I20260811 22:31:23.154965 3384 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:23.155061 3384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:23.159502 3384 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:31:23.161854 3411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:23.163331 3384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:23.163471 3384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "d6751f3aff9a4262b61ad35a0c8ec352"
format_stamp: "Formatted at 2026-08-11 22:31:22 on dist-test-slave-8hhm"
server_key: "17c1ccd2c8b330b4f2f82393319fc3c6"
server_key_iv: "1354499a1e5e8af3e98149780525f0c7"
server_key_version: "encryptionkey@0"
I20260811 22:31:23.164147 3384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:23.199378 3384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:23.202986 3384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:23.203220 3384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:23.203879 3384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:23.204811 3384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:23.204886 3384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:23.204963 3384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:23.205027 3384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:23.214572 3384 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:36129
I20260811 22:31:23.214604 3620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:36129 every 8 connection(s)
I20260811 22:31:23.215641 3384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:23.224442 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 3384
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:31:23.227468 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:23.217699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:60817 (local address 127.18.211.190:45711)
0811 22:31:23.217871 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.217876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.218779 (+ 903us) server_negotiation.cc:407] Connection header received
0811 22:31:23.220182 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.220184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.220233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.220312 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.221528 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.222054 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.223205 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.223414 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.223490 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.227367 (+ 3877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:60817: BlockingRecv error: recv got EOF from 127.18.211.131:60817 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:31:23.228168 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.218132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:60817)
0811 22:31:23.218620 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:23.218665 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:23.219969 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:23.220481 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:23.220488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:23.220867 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:23.221388 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.221403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.222177 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.222179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.223065 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.223074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.223548 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.223551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.223712 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.224314 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:23.224331 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:23.227266 (+ 2935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":125,"threads_started":1}
W20260811 22:31:23.228807 3621 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:45711 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.211.190:45711: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:23.234337 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.229368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:53397)
0811 22:31:23.229482 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:23.229496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:23.229571 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:23.229836 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:23.229838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:23.230017 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:23.230105 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.230110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.230805 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.230809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.231322 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.231330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.231791 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.231794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.231887 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.232308 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:23.232319 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:23.234196 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 22:31:23.234426 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:23.229421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:53397 (local address 127.18.211.190:45711)
0811 22:31:23.229540 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.229544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.229556 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:31:23.229693 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.229696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.229741 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.229819 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.230219 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.230698 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.231441 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.231659 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.231788 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.234307 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:53397: BlockingRecv error: recv got EOF from 127.18.211.131:53397 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:31:23.235625 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.225659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.18.211.190:45711)
0811 22:31:23.225919 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.225922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.225933 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:31:23.226002 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.226005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.226053 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.226161 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.226944 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.227670 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.228593 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.228889 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.229005 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.230594 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:23.230603 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:23.230626 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:23.230656 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:23.232627 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:23.233669 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:23.233675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:23.233680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:23.233839 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:23.234177 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:23.234180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:23.234182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:23.234659 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:23.234893 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:23.235220 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 22:31:23.235494 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:23.241259 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.235066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:48511)
0811 22:31:23.235232 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:23.235247 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:23.235319 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:23.235596 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:23.235599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:23.235757 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:23.235835 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.235841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.236562 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.236565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.237368 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.237380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.238063 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.238066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.238122 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.238763 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:23.238778 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:23.241078 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 22:31:23.241503 3158 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:23.235161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:48511 (local address 127.18.211.190:45711)
0811 22:31:23.235285 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.235288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.235301 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:23.235444 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.235447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.235495 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.235581 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.235933 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.236408 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.237474 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.237707 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.237816 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.241387 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:48511: BlockingRecv error: recv got EOF from 127.18.211.131:48511 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:31:23.242871 3621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:23 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:23.836233 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:23.830529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:47101)
0811 22:31:23.830808 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:23.830821 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:23.830912 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:23.831169 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:23.831172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:23.831350 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:23.831459 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.831464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.832345 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.832348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.832920 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:23.832929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.833674 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.833677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:23.833751 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.834336 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:23.834349 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:23.836113 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
W20260811 22:31:23.836385 3631 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:23.830572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:47101 (local address 127.18.211.190:45711)
0811 22:31:23.830854 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:23.830858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:23.830872 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:23.831003 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:23.831005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:23.831058 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:23.831137 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:23.831661 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.832221 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.833055 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:23.833431 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:23.833542 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:23.836249 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:47101: BlockingRecv error: recv got EOF from 127.18.211.129:47101 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
Aug 11 22:31:24 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:24 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:24.030828 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:24.024420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:42057)
0811 22:31:24.024776 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:24.024795 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:24.024926 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:24.025354 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:24.025357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:24.025604 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:24.025730 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:24.025737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.026599 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.026604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:24.027286 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:24.027294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.027842 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.027846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:24.027970 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:24.028525 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:24.028537 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:24.030710 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":90,"threads_started":1}
W20260811 22:31:24.031036 3631 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:24.024577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:42057 (local address 127.18.211.190:45711)
0811 22:31:24.024758 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:24.024763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:24.024877 (+ 114us) server_negotiation.cc:407] Connection header received
0811 22:31:24.025104 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:24.025109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:24.025166 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:24.025329 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:24.025864 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.026473 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.027421 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.027686 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.027840 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:24.030859 (+ 3019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:42057: BlockingRecv error: recv got EOF from 127.18.211.130:42057 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 22:31:24 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:24 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:24.249944 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:24.243558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:41167)
0811 22:31:24.243925 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:24.243943 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:24.244099 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:24.244382 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:24.244385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:24.244639 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:24.244766 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:24.244772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.245653 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.245657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:24.246338 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:24.246348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.246799 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.246803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:24.246888 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:24.247426 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:24.247438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:24.249745 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":103,"threads_started":1}
W20260811 22:31:24.249984 3631 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:24.243766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:41167 (local address 127.18.211.190:45711)
0811 22:31:24.243925 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:24.243931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:24.244042 (+ 111us) server_negotiation.cc:407] Connection header received
0811 22:31:24.244214 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:24.244218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:24.244271 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:24.244358 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:24.244880 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.245552 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.246453 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.246685 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.246794 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:24.249880 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:41167: BlockingRecv error: recv got EOF from 127.18.211.131:41167 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 22:31:24 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:24 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:24.843531 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:24.837309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:47041)
0811 22:31:24.837600 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:24.837614 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:24.837707 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:24.838021 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:24.838023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:24.838223 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:24.838324 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:24.838330 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.839403 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.839412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:24.840318 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:24.840333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.840812 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.840815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:24.840874 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:24.841420 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:24.841431 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:24.843368 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
W20260811 22:31:24.843616 3639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:24.837391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:47041 (local address 127.18.211.190:45711)
0811 22:31:24.837785 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:24.837788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:24.837802 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:24.837843 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:24.837854 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:24.837898 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:24.837985 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:24.838487 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.839239 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.840433 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:24.840661 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:24.840762 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:24.843507 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:47041: BlockingRecv error: recv got EOF from 127.18.211.129:47041 (error 108)
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":233,"threads_started":1}
Aug 11 22:31:25 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:25 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:25.037535 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:25.031749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:37577)
0811 22:31:25.032026 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:25.032039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:25.032139 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:25.032358 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:25.032360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:25.032540 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:25.032635 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:25.032641 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.033614 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.033616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:25.034170 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:25.034178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.034678 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.034681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:25.034742 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:25.035321 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:25.035332 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:25.037400 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
W20260811 22:31:25.037766 3639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:25.031780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:37577 (local address 127.18.211.190:45711)
0811 22:31:25.031935 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:25.031938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:25.032111 (+ 173us) server_negotiation.cc:407] Connection header received
0811 22:31:25.032221 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:25.032224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:25.032271 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:25.032341 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:25.032784 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.033506 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.034300 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.034566 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.034665 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:25.037606 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:37577: BlockingRecv error: recv got EOF from 127.18.211.130:37577 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 22:31:25 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:25 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:25.257102 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:25.250912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:59797)
0811 22:31:25.251291 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:25.251310 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:25.251463 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:25.251767 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:25.251770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:25.251994 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:25.252091 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:25.252097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.252877 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.252882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:25.253522 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:25.253533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.254019 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.254023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:25.254109 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:25.254636 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:25.254648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:25.256872 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":166,"threads_started":1}
W20260811 22:31:25.257177 3639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:25.251122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:59797 (local address 127.18.211.190:45711)
0811 22:31:25.251291 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:25.251297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:25.251398 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:31:25.251570 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:25.251573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:25.251625 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:25.251743 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:25.252223 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.252761 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.253646 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.253894 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.254003 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:25.257056 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:59797: BlockingRecv error: recv got EOF from 127.18.211.131:59797 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 22:31:25 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:25 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:25.850457 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:25.844505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:40799)
0811 22:31:25.844758 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:25.844773 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:25.844890 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:25.845248 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:25.845251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:25.845466 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:25.845591 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:25.845602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.846519 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.846524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:25.847190 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:25.847200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.847660 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.847663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:25.847725 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:25.848315 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:25.848328 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:25.850335 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":73,"threads_started":1}
W20260811 22:31:25.850642 3644 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:25.844626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:40799 (local address 127.18.211.190:45711)
0811 22:31:25.844875 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:25.844879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:25.844893 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:25.844997 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:25.845002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:25.845066 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:25.845178 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:25.845743 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.846362 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.847333 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:25.847555 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:25.847641 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:25.850471 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:40799: BlockingRecv error: recv got EOF from 127.18.211.129:40799 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
Aug 11 22:31:26 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:26 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:26.044417 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:26.038387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:57811)
0811 22:31:26.038695 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:26.038709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:26.038817 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:26.039037 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:26.039039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:26.039224 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:26.039321 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:26.039326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.040214 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.040239 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:26.041080 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:26.041092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.041599 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.041602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:26.041664 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:26.042225 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:26.042236 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:26.044287 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":124,"threads_started":1}
W20260811 22:31:26.044656 3644 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:26.038462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:57811 (local address 127.18.211.190:45711)
0811 22:31:26.038628 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:26.038632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:26.038795 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:31:26.038901 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:26.038903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:26.038950 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:26.039030 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:26.039457 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.040092 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.041236 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.041482 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.041589 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:26.044430 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:57811: BlockingRecv error: recv got EOF from 127.18.211.130:57811 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 22:31:26 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:26 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:26.263633 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:26.257996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:42215)
0811 22:31:26.258355 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:26.258369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:26.258476 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:26.258726 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:26.258728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:26.258920 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:26.259021 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:26.259026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.259780 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.259782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:26.260352 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:26.260361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.260800 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.260802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:26.260863 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:26.261396 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:26.261408 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:26.263485 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":164,"threads_started":1}
W20260811 22:31:26.263785 3644 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:26.258063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:42215 (local address 127.18.211.190:45711)
0811 22:31:26.258212 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:26.258216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:26.258446 (+ 230us) server_negotiation.cc:407] Connection header received
0811 22:31:26.258559 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:26.258562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:26.258609 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:26.258694 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:26.259136 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.259669 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.260462 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.260698 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.260774 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:26.263643 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:42215: BlockingRecv error: recv got EOF from 127.18.211.131:42215 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 22:31:26 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:26 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:26.856653 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:26.851328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:58129)
0811 22:31:26.851603 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:26.851623 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:26.851756 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:26.852018 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:26.852020 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:26.852228 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:26.852326 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:26.852332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.853077 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.853079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:26.853654 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:26.853662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.854121 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.854123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:26.854185 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:26.854672 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:26.854683 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:26.856521 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
W20260811 22:31:26.856779 3649 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:26.851421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:58129 (local address 127.18.211.190:45711)
0811 22:31:26.851678 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:26.851709 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:26.851751 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:31:26.851846 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:26.851849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:26.851903 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:26.851981 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:26.852433 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.852964 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.853756 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:26.853977 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:26.854056 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:26.856667 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:58129: BlockingRecv error: recv got EOF from 127.18.211.129:58129 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":81,"threads_started":1}
Aug 11 22:31:27 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:27 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:27.050938 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:27.045203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:49245)
0811 22:31:27.045516 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:27.045530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:27.045629 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:27.045948 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:27.045951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:27.046179 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:27.046276 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:27.046281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.047060 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.047065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:27.047619 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:27.047627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.048118 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.048122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:27.048230 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:27.048816 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:27.048828 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:27.050816 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":95,"threads_started":1}
W20260811 22:31:27.051121 3649 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:27.045299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:49245 (local address 127.18.211.190:45711)
0811 22:31:27.045451 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:27.045455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:27.045581 (+ 126us) server_negotiation.cc:407] Connection header received
0811 22:31:27.045752 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:27.045756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:27.045829 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:27.045919 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:27.046396 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.046942 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.047752 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.047995 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.048101 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:27.050975 (+ 2874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:49245: BlockingRecv error: recv got EOF from 127.18.211.130:49245 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 22:31:27 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:27 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:31:27.271047 3649 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:27.264935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:39663 (local address 127.18.211.190:45711)
0811 22:31:27.265103 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:27.265109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:27.265361 (+ 252us) server_negotiation.cc:407] Connection header received
0811 22:31:27.265528 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:27.265531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:27.265602 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:27.265721 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:27.266235 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.266776 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.267682 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.267920 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.268036 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:27.270935 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:39663: BlockingRecv error: recv got EOF from 127.18.211.131:39663 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:31:27.270980 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:27.264848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:39663)
0811 22:31:27.265253 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:27.265269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:27.265434 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:27.265743 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:27.265746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:27.265974 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:27.266108 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:27.266116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.266892 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.266897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:27.267546 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:27.267564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.268058 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.268062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:27.268146 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:27.268729 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:27.268741 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:27.270800 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":101,"threads_started":1}
Aug 11 22:31:27 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:27 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:27.863066 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:27.857687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:39121)
0811 22:31:27.857969 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:27.857987 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:27.858094 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:27.858368 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:27.858370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:27.858559 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:27.858652 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:27.858657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.859413 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.859415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:27.859969 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:27.859976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.860403 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.860406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:27.860465 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:27.861046 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:27.861059 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:27.862935 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":114,"threads_started":1}
W20260811 22:31:27.863247 3655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:27.857789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:39121 (local address 127.18.211.190:45711)
0811 22:31:27.858022 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:27.858025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:27.858051 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:31:27.858198 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:27.858201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:27.858245 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:27.858328 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:27.858774 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.859315 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.860077 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:27.860301 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:27.860380 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:27.863061 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:39121: BlockingRecv error: recv got EOF from 127.18.211.129:39121 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":86,"threads_started":1}
Aug 11 22:31:28 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:28 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:28.057997 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:28.051990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:38507)
0811 22:31:28.052352 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:28.052371 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:28.052485 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:28.052800 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:28.052802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:28.053032 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:28.053132 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:28.053138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.054079 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.054084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:28.054705 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:28.054714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.055200 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.055204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:28.055291 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:28.055816 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:28.055828 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:28.057817 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":103,"threads_started":1}
W20260811 22:31:28.058123 3655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:28.052287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:38507 (local address 127.18.211.190:45711)
0811 22:31:28.052454 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:28.052459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:28.052475 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:28.052619 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:28.052623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:28.052677 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:28.052769 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:28.053277 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.053960 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.054833 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.055072 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.055182 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:28.057963 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:38507: BlockingRecv error: recv got EOF from 127.18.211.130:38507 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 22:31:28 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:28 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:28.281435 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:28.271884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:43971)
0811 22:31:28.272233 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:28.272247 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:28.272342 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:28.272593 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:28.272595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:28.272796 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:28.272891 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:28.272896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.273763 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.273766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:28.274423 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:28.274434 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.274844 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.274846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:28.274904 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:28.279241 (+ 4337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:28.279254 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:28.281254 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":87,"threads_started":1}
W20260811 22:31:28.281490 3655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:28.271993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:43971 (local address 127.18.211.190:45711)
0811 22:31:28.272211 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:28.272216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:28.272318 (+ 102us) server_negotiation.cc:407] Connection header received
0811 22:31:28.272423 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:28.272426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:28.272473 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:28.272562 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:28.272995 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.273664 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.274523 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.274740 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.274823 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:28.281382 (+ 6559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:43971: BlockingRecv error: recv got EOF from 127.18.211.131:43971 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:31:28 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:28 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:28.869668 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:28.863865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:52067)
0811 22:31:28.864160 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:28.864174 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:28.864266 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:28.864538 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:28.864540 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:28.864798 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:28.864896 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:28.864902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.865787 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.865790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:28.866379 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:28.866387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.866850 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.866853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:28.866911 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:28.867401 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:28.867413 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:28.869487 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":93,"threads_started":1}
W20260811 22:31:28.869763 3660 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:28.863926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:52067 (local address 127.18.211.190:45711)
0811 22:31:28.864234 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:28.864238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:28.864251 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:28.864354 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:28.864356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:28.864403 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:28.864497 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:28.865045 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.865690 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.866483 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:28.866717 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:28.866809 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:28.869630 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:52067: BlockingRecv error: recv got EOF from 127.18.211.129:52067 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":157,"threads_started":1}
Aug 11 22:31:29 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:29 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:29.064344 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:29.058841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:33847)
0811 22:31:29.059217 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:29.059234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:29.059377 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:29.059622 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:29.059624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:29.059815 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:29.059911 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:29.059916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.060634 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.060636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:29.061178 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:29.061185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.061657 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.061659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:29.061722 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:29.062207 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:29.062220 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:29.064207 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":85,"threads_started":1}
W20260811 22:31:29.064518 3660 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:29.058908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:33847 (local address 127.18.211.190:45711)
0811 22:31:29.059064 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:29.059068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:29.059316 (+ 248us) server_negotiation.cc:407] Connection header received
0811 22:31:29.059468 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:29.059470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:29.059515 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:29.059583 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:29.060012 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.060538 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.061346 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.061561 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.061635 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:29.064380 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:33847: BlockingRecv error: recv got EOF from 127.18.211.130:33847 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 22:31:29 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:29 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:29.288538 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:29.282372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:48965)
0811 22:31:29.282742 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:29.282762 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:29.282898 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:29.283353 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:29.283356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:29.283628 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:29.283774 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:29.283782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.284862 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.284865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:29.285442 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:29.285450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.285892 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.285894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:29.285953 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:29.286536 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:29.286552 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:29.288422 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":88,"threads_started":1}
W20260811 22:31:29.288700 3660 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:29.282503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:48965 (local address 127.18.211.190:45711)
0811 22:31:29.282688 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:29.282694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:29.282840 (+ 146us) server_negotiation.cc:407] Connection header received
0811 22:31:29.283068 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:29.283072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:29.283170 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:29.283289 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:29.283904 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.284752 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.285556 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.285788 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.285875 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:29.288572 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:48965: BlockingRecv error: recv got EOF from 127.18.211.131:48965 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:31:29 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:29 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:29.876376 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:29.870464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:38439)
0811 22:31:29.870792 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:29.870805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:29.870986 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:29.871366 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:29.871368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:29.871561 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:29.871672 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:29.871677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.872494 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.872496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:29.873054 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:29.873063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.873671 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.873674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:29.873743 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:29.874362 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:29.874384 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:29.876259 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":96,"threads_started":1}
W20260811 22:31:29.876566 3666 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:29.870560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:38439 (local address 127.18.211.190:45711)
0811 22:31:29.870809 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:29.870812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:29.870937 (+ 125us) server_negotiation.cc:407] Connection header received
0811 22:31:29.871183 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:29.871187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:29.871244 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:29.871325 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:29.871803 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.872397 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.873246 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:29.873544 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:29.873637 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:29.876414 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:38439: BlockingRecv error: recv got EOF from 127.18.211.129:38439 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":107,"threads_started":1}
Aug 11 22:31:30 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:30 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:30.071663 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:30.065473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:45637)
0811 22:31:30.065834 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:30.065848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:30.065955 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:30.066247 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:30.066249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:30.066444 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:30.066540 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:30.066545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.067343 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.067347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:30.067915 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:30.067924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.068472 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.068475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:30.068544 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:30.069164 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:30.069181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:30.071471 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":130,"threads_started":1}
W20260811 22:31:30.071888 3666 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:30.065537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:45637 (local address 127.18.211.190:45711)
0811 22:31:30.065711 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:30.065715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:30.065929 (+ 214us) server_negotiation.cc:407] Connection header received
0811 22:31:30.066033 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:30.066036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:30.066093 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:30.066198 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:30.066641 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.067229 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.068129 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.068356 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.068489 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:30.071680 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:45637: BlockingRecv error: recv got EOF from 127.18.211.130:45637 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 22:31:30 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:30 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:30.296048 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:30.289562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:59171)
0811 22:31:30.290009 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:30.290025 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:30.290178 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:30.290510 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:30.290513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:30.290761 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:30.290860 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:30.290867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.291756 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.291763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:30.292394 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:30.292403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.292944 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.292947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:30.293022 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:30.293715 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:30.293748 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:30.295921 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":119,"threads_started":1}
W20260811 22:31:30.296236 3666 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:30.289731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:59171 (local address 127.18.211.190:45711)
0811 22:31:30.289932 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:30.289938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:30.290118 (+ 180us) server_negotiation.cc:407] Connection header received
0811 22:31:30.290329 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:30.290332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:30.290384 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:30.290480 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:30.290999 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.291615 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.292538 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.292795 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.292926 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:30.296052 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:59171: BlockingRecv error: recv got EOF from 127.18.211.131:59171 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 22:31:30 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:30 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:30.884331 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:30.877330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:43301)
0811 22:31:30.877615 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:30.877630 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:30.877725 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:30.877994 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:30.877996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:30.878200 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:30.878297 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:30.878302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.879349 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.879353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:30.880243 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:30.880255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.880705 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.880712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:30.880802 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:30.881536 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:30.881557 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:30.884145 (+ 2588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
W20260811 22:31:30.884423 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:30.877489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:43301 (local address 127.18.211.190:45711)
0811 22:31:30.877713 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:30.877716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:30.877732 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:31:30.877815 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:30.877818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:30.877879 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:30.877959 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:30.878421 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.879206 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.880353 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:30.880577 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:30.880676 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:30.884288 (+ 3612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:43301: BlockingRecv error: recv got EOF from 127.18.211.129:43301 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
Aug 11 22:31:31 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:31 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:31.078501 3675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:31.072529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:55537)
0811 22:31:31.072854 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:31.072867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:31.073003 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:31.073446 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:31.073455 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:31.073648 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:31.073749 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:31.073754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.074671 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.074676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:31.075269 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:31.075278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.075773 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.075776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:31.075865 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:31.076418 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:31.076429 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:31.078380 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":114,"threads_started":1}
W20260811 22:31:31.078661 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:31.072618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:55537 (local address 127.18.211.190:45711)
0811 22:31:31.072801 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:31.072806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:31.072945 (+ 139us) server_negotiation.cc:407] Connection header received
0811 22:31:31.073108 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:31.073111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:31.073168 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:31.073280 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:31.073878 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.074550 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.075398 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.075637 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.075767 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:31.078532 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:55537: BlockingRecv error: recv got EOF from 127.18.211.130:55537 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 22:31:31 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:31 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:31.302922 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:31.296986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:36929)
0811 22:31:31.297291 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:31.297305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:31.297409 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:31.297731 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:31.297734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:31.297933 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:31.298072 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:31.298079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.298906 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.298909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:31.299527 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:31.299536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.300063 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.300066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:31.300133 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:31.300743 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:31.300755 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:31.302801 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":94,"threads_started":1}
W20260811 22:31:31.303145 3674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:31.297042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:36929 (local address 127.18.211.190:45711)
0811 22:31:31.297266 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:31.297271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:31.297385 (+ 114us) server_negotiation.cc:407] Connection header received
0811 22:31:31.297502 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:31.297506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:31.297556 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:31.297631 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:31.298239 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.298786 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.299678 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.299911 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.300000 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:31.303000 (+ 3000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:36929: BlockingRecv error: recv got EOF from 127.18.211.131:36929 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 22:31:31 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:31 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:31.890686 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:31.885201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:48781)
0811 22:31:31.885544 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:31.885558 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:31.885661 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:31.885900 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:31.885902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:31.886098 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:31.886202 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:31.886207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.886947 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.886950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:31.887503 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:31.887521 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.887937 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.887940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:31.888004 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:31.888609 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:31.888623 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:31.890563 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
W20260811 22:31:31.890852 3681 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:31.885296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:48781 (local address 127.18.211.190:45711)
0811 22:31:31.885550 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:31.885554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:31.885609 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:31:31.885729 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:31.885731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:31.885775 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:31.885855 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:31.886311 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.886839 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.887618 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:31.887836 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:31.887910 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:31.890712 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:48781: BlockingRecv error: recv got EOF from 127.18.211.129:48781 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
Aug 11 22:31:32 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:32 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:32.085729 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:32.079560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:60919)
0811 22:31:32.079882 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:32.079898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:32.080031 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:32.080360 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:32.080363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:32.080605 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:32.080708 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:32.080715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.081631 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.081637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:32.082220 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:32.082230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.082830 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.082834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:32.082933 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:32.083467 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:32.083480 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:32.085590 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":104,"threads_started":1}
W20260811 22:31:32.085875 3681 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:32.079649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:60919 (local address 127.18.211.190:45711)
0811 22:31:32.079807 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:32.079812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:32.079978 (+ 166us) server_negotiation.cc:407] Connection header received
0811 22:31:32.080167 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:32.080177 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:32.080230 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:32.080332 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:32.080849 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.081507 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.082385 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.082682 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.082826 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:32.085747 (+ 2921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:60919: BlockingRecv error: recv got EOF from 127.18.211.130:60919 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 22:31:32 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:32 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:32.309724 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:32.303832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:33107)
0811 22:31:32.304233 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:32.304250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:32.304367 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:32.304604 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:32.304607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:32.304818 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:32.304931 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:32.304938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.305799 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.305803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:32.306410 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:32.306421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.306888 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.306892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:32.306965 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:32.307502 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:32.307526 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:32.309576 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":187,"threads_started":1}
W20260811 22:31:32.309816 3681 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:32.303899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:33107 (local address 127.18.211.190:45711)
0811 22:31:32.304035 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:32.304039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:32.304328 (+ 289us) server_negotiation.cc:407] Connection header received
0811 22:31:32.304440 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:32.304442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:32.304488 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:32.304557 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:32.305045 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.305670 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.306539 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.306757 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.306834 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:32.309718 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:33107: BlockingRecv error: recv got EOF from 127.18.211.131:33107 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 22:31:32 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:32 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:32.896927 3684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:32.891537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:43471)
0811 22:31:32.891848 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:32.891863 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:32.891952 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:32.892168 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:32.892170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:32.892364 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:32.892463 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:32.892469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.893198 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.893201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:32.893841 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:32.893850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.894289 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.894292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:32.894358 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:32.894878 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:32.894891 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:32.896803 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
W20260811 22:31:32.897084 3688 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:32.891643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:43471 (local address 127.18.211.190:45711)
0811 22:31:32.891922 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:32.891925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:32.891938 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:32.892033 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:32.892036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:32.892078 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:32.892153 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:32.892566 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.893092 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.893948 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:32.894174 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:32.894261 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:32.896951 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:43471: BlockingRecv error: recv got EOF from 127.18.211.129:43471 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":101,"threads_started":1}
Aug 11 22:31:33 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:33 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:33.092420 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:33.086792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:38629)
0811 22:31:33.087080 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:33.087094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:33.087200 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:33.087475 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:33.087477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:33.087688 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:33.087791 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:33.087797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.088516 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.088518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:33.089082 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:33.089115 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.089633 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.089636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:33.089704 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:33.090329 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:33.090341 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:33.092302 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":95,"threads_started":1}
W20260811 22:31:33.092567 3688 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:33.086872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:38629 (local address 127.18.211.190:45711)
0811 22:31:33.087030 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:33.087033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:33.087172 (+ 139us) server_negotiation.cc:407] Connection header received
0811 22:31:33.087290 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:33.087293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:33.087339 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:33.087416 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:33.087902 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.088422 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.089287 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.089505 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.089619 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:33.092447 (+ 2828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:38629: BlockingRecv error: recv got EOF from 127.18.211.130:38629 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 22:31:33 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:33 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:33.321554 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:33.310768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:43801)
0811 22:31:33.311201 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:33.311214 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:33.311303 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:33.311628 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:33.311634 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:33.311864 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:33.311962 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:33.311968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.313103 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.313106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:33.313740 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:33.313749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.314221 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.314223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:33.314286 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:33.319480 (+ 5194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:33.319493 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:33.321432 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":120,"threads_started":1}
W20260811 22:31:33.321847 3688 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:33.311018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:43801 (local address 127.18.211.190:45711)
0811 22:31:33.311238 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:33.311246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:33.311269 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:31:33.311392 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:33.311395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:33.311456 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:33.311569 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:33.312080 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.312995 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.313859 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.314109 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.314198 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:33.321608 (+ 7410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:43801: BlockingRecv error: recv got EOF from 127.18.211.131:43801 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 22:31:33 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:33 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:33.903492 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:33.897801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:48549)
0811 22:31:33.898086 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:33.898099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:33.898202 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:33.898444 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:33.898445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:33.898636 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:33.898734 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:33.898739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.899498 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.899500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:33.900036 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:33.900044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.900510 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.900513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:33.900578 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:33.901252 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:33.901271 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:33.903314 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":101,"threads_started":1}
W20260811 22:31:33.903568 3693 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:33.897866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:48549 (local address 127.18.211.190:45711)
0811 22:31:33.898116 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:33.898119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:33.898167 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:31:33.898283 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:33.898285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:33.898330 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:33.898429 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:33.898834 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.899389 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.900133 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:33.900399 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:33.900476 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:33.903443 (+ 2967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:48549: BlockingRecv error: recv got EOF from 127.18.211.129:48549 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
Aug 11 22:31:34 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:34 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:34.098565 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:34.093344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:34921)
0811 22:31:34.093616 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:34.093629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:34.093727 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:34.093965 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:34.093967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:34.094189 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:34.094289 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:34.094295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.095075 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.095077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:34.095636 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:34.095643 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.096091 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.096094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:34.096153 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:34.096611 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:34.096622 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:34.098416 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":90,"threads_started":1}
W20260811 22:31:34.098693 3693 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:34.093371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:34921 (local address 127.18.211.190:45711)
0811 22:31:34.093513 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:34.093516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:34.093705 (+ 189us) server_negotiation.cc:407] Connection header received
0811 22:31:34.093808 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:34.093810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:34.093855 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:34.093925 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:34.094409 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.094974 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.095742 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.095954 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.096028 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:34.098567 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:34921: BlockingRecv error: recv got EOF from 127.18.211.130:34921 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 22:31:34 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:34 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:34.328591 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:34.322335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:55571)
0811 22:31:34.322720 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:34.322736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:34.323019 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:34.323318 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:34.323321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:34.323554 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:34.323649 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:34.323655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.324639 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.324643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:34.325392 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:34.325404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.325924 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.325928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:34.326016 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:34.326511 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:34.326523 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:34.328393 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":100,"threads_started":1}
W20260811 22:31:34.328629 3693 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:34.322485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:55571 (local address 127.18.211.190:45711)
0811 22:31:34.322691 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:34.322697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:34.322941 (+ 244us) server_negotiation.cc:407] Connection header received
0811 22:31:34.323139 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:34.323142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:34.323204 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:34.323297 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:34.323777 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.324407 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.325528 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.325797 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.325909 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:34.328523 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:55571: BlockingRecv error: recv got EOF from 127.18.211.131:55571 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 22:31:34 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:34 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:34.910244 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:34.904479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:59911)
0811 22:31:34.904776 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:34.904788 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:34.904896 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:34.905313 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:34.905316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:34.905503 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:34.905602 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:34.905607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.906445 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.906449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:34.907161 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:34.907173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.907720 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.907723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:34.907785 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:34.908285 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:34.908297 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:34.910115 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":71,"threads_started":1}
W20260811 22:31:34.910418 3697 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:34.904593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:59911 (local address 127.18.211.190:45711)
0811 22:31:34.904820 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:34.904824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:34.904857 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:31:34.905031 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:34.905035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:34.905109 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:34.905197 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:34.905710 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.906333 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.907296 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:34.907609 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:34.907692 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:34.910261 (+ 2569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:59911: BlockingRecv error: recv got EOF from 127.18.211.129:59911 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
Aug 11 22:31:35 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:35 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:35.104763 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:35.099400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:58321)
0811 22:31:35.099669 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:35.099682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:35.099809 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:35.100045 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:35.100047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:35.100247 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:35.100345 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:35.100350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.101115 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.101117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:35.101689 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:35.101697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.102122 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.102125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:35.102195 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:35.102800 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:35.102811 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:35.104637 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":95,"threads_started":1}
W20260811 22:31:35.104933 3697 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:35.099523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:58321 (local address 127.18.211.190:45711)
0811 22:31:35.099669 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:35.099672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:35.099826 (+ 154us) server_negotiation.cc:407] Connection header received
0811 22:31:35.099879 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:35.099882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:35.099931 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:35.100007 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:35.100457 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.100997 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.101787 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.101998 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.102105 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:35.104803 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:58321: BlockingRecv error: recv got EOF from 127.18.211.130:58321 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 22:31:35 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:35 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:35.335121 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:35.329547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:51281)
0811 22:31:35.329938 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:35.329955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:35.330107 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:35.330399 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:35.330401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:35.330694 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:35.330840 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:35.330847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.331566 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.331568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:35.332122 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:35.332129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.332538 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.332541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:35.332601 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:35.333120 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:35.333131 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:35.335002 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":179,"threads_started":1}
W20260811 22:31:35.335268 3697 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:35.329664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:51281 (local address 127.18.211.190:45711)
0811 22:31:35.329837 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:35.329843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:35.330042 (+ 199us) server_negotiation.cc:407] Connection header received
0811 22:31:35.330215 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:35.330219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:35.330277 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:35.330372 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:35.330945 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.331472 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.332234 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.332448 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.332519 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:35.335140 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:51281: BlockingRecv error: recv got EOF from 127.18.211.131:51281 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:31:35 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:35 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:35.916908 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:35.911162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:40525)
0811 22:31:35.911475 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:35.911489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:35.911595 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:35.911880 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:35.911882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:35.912096 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:35.912199 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:35.912205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.912934 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.912937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:35.913599 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:35.913611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.914147 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.914151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:35.914258 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:35.914894 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:35.914907 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:35.916762 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":114,"threads_started":1}
W20260811 22:31:35.917088 3705 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:35.911225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:40525 (local address 127.18.211.190:45711)
0811 22:31:35.911477 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:35.911481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:35.911562 (+ 81us) server_negotiation.cc:407] Connection header received
0811 22:31:35.911680 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:35.911683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:35.911742 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:35.911824 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:35.912310 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.912829 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.913709 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:35.913979 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:35.914088 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:35.916920 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:40525: BlockingRecv error: recv got EOF from 127.18.211.129:40525 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
Aug 11 22:31:36 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:36 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:36.111776 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:36.105761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:42791)
0811 22:31:36.106092 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:36.106112 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:36.106233 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:36.106474 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:36.106477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:36.106692 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:36.106795 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:36.106800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.107691 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.107694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:36.108321 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:36.108330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.108759 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.108763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:36.108830 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:36.109570 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:36.109583 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:36.111638 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":144,"threads_started":1}
W20260811 22:31:36.111979 3705 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:36.105899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:42791 (local address 127.18.211.190:45711)
0811 22:31:36.106075 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:36.106078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:36.106201 (+ 123us) server_negotiation.cc:407] Connection header received
0811 22:31:36.106305 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:36.106307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:36.106351 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:36.106424 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:36.107020 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.107566 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.108438 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.108649 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.108725 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:36.111834 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:42791: BlockingRecv error: recv got EOF from 127.18.211.130:42791 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 22:31:36 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:36 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:36.342377 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:36.336048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:36749)
0811 22:31:36.336491 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:36.336507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:36.336712 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:36.336988 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:36.336991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:36.337325 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:36.337437 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:36.337443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.338382 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.338388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:36.339080 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:36.339090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.339586 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.339590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:36.339675 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:36.340243 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:36.340254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:36.342248 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":97,"threads_started":1}
W20260811 22:31:36.342557 3705 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:36.336287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:36749 (local address 127.18.211.190:45711)
0811 22:31:36.336495 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:36.336503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:36.336656 (+ 153us) server_negotiation.cc:407] Connection header received
0811 22:31:36.336814 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:36.336817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:36.336872 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:36.336964 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:36.337565 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.338276 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.339216 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.339450 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.339572 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:36.342393 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:36749: BlockingRecv error: recv got EOF from 127.18.211.131:36749 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 22:31:36 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:36 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:36.923403 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:36.917774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:43135)
0811 22:31:36.918108 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:36.918123 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:36.918216 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:36.918458 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:36.918460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:36.918664 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:36.918760 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:36.918766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.919562 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.919565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:36.920132 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:36.920141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.920631 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.920635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:36.920715 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:36.921329 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:36.921341 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:36.923281 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":123,"threads_started":1}
W20260811 22:31:36.923602 3712 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:36.917841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:43135 (local address 127.18.211.190:45711)
0811 22:31:36.918193 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:36.918196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:36.918209 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:31:36.918299 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:36.918301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:36.918346 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:36.918426 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:36.918901 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.919456 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.920258 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:36.920495 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:36.920580 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:36.923440 (+ 2860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:43135: BlockingRecv error: recv got EOF from 127.18.211.129:43135 (error 108)
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":124,"threads_started":1}
Aug 11 22:31:37 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:37 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:37.119029 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:37.112887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:57393)
0811 22:31:37.113195 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:37.113242 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:37.113401 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:37.113683 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:37.113685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:37.113881 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:37.113980 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:37.113985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.114762 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.114765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:37.115371 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:37.115382 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.115813 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.115816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:37.115890 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:37.116529 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:37.116554 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:37.118893 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":132,"threads_started":1}
W20260811 22:31:37.119194 3712 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:37.112961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:57393 (local address 127.18.211.190:45711)
0811 22:31:37.113122 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:37.113126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:37.113352 (+ 226us) server_negotiation.cc:407] Connection header received
0811 22:31:37.113499 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:37.113501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:37.113547 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:37.113622 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:37.114096 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.114624 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.115491 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.115703 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.115779 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:37.119030 (+ 3251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:57393: BlockingRecv error: recv got EOF from 127.18.211.130:57393 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 22:31:37 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:37 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.131@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:37.349723 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:37.343380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.131:37967)
0811 22:31:37.343662 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:37.343676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:37.343784 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:37.344143 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:37.344145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:37.344370 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:37.344513 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:37.344521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.345655 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.345659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:37.346342 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:37.346367 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.346850 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.346854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:37.346943 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:37.347528 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:37.347540 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:37.349534 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":106,"threads_started":1}
W20260811 22:31:37.349766 3712 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:37.343547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:37967 (local address 127.18.211.190:45711)
0811 22:31:37.343692 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:37.343696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:37.343733 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:31:37.343921 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:37.343924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:37.344017 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:37.344109 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:37.344705 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.345545 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.346490 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.346726 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.346841 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:37.349658 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.131:37967: BlockingRecv error: recv got EOF from 127.18.211.131:37967 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 22:31:37 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:37 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.129@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:37.929788 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:37.924365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.129:36423)
0811 22:31:37.924684 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:37.924697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:37.924796 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:37.925098 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:37.925100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:37.925313 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:37.925411 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:37.925417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.926152 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.926155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:37.926715 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:37.926724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.927139 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.927141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:37.927202 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:37.927703 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:37.927714 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:37.929670 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":114,"threads_started":1}
W20260811 22:31:37.930003 3716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:37.924427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:36423 (local address 127.18.211.190:45711)
0811 22:31:37.924694 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:37.924698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:37.924769 (+ 71us) server_negotiation.cc:407] Connection header received
0811 22:31:37.924873 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:37.924875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:37.924919 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:37.924996 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:37.925520 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.926052 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.926821 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:37.927041 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:37.927116 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:37.929814 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.129:36423: BlockingRecv error: recv got EOF from 127.18.211.129:36423 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
Aug 11 22:31:38 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:31:38 dist-test-slave-8hhm krb5kdc[1841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.211.130@KRBTEST.COM for oryx/127.18.211.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:31:38.125653 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:38.120003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45711 (local address 127.18.211.130:48651)
0811 22:31:38.120368 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:38.120381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:38.120513 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:38.120815 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:38.120818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:38.121041 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:38.121131 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:38.121137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.121951 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.121956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:38.122523 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:38.122531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.123002 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.123006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:38.123095 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:38.123633 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:38.123644 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:38.125539 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.211.190:45711: Server oryx/127.18.211.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":107,"threads_started":1}
W20260811 22:31:38.125801 3716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:38.120111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:48651 (local address 127.18.211.190:45711)
0811 22:31:38.120268 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:38.120273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:38.120463 (+ 190us) server_negotiation.cc:407] Connection header received
0811 22:31:38.120626 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:38.120630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:38.120695 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:38.120789 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:38.121303 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.121836 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.122651 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.122882 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.122997 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:38.125682 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.211.130:48651: BlockingRecv error: recv got EOF from 127.18.211.130:48651 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:38.224783 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 2894
I20260811 22:31:38.245936 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 3162
I20260811 22:31:38.266513 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 3384
I20260811 22:31:38.284679 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 2777
2026-08-11T22:31:38Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18535 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T22:31:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:38Z Disabled control of system clock
I20260811 22:31:38.328677 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43797
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:43797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:31:38.444239 3726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:38.444530 3726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:38.444572 3726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:38.448648 3726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:38.448709 3726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:38.448722 3726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:38.448756 3726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:38.448772 3726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:38.453636 3726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:43797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:38.454828 3726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:38.456264 3726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:38.465273 3737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:38.465256 3736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:38.465255 3745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:38.465343 3726 server_base.cc:1034] running on GCE node
I20260811 22:31:38.466110 3726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:38.467237 3726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:38.468428 3726 hybrid_clock.cc:648] HybridClock initialized: now 1786487498468408 us; error 35 us; skew 500 ppm
I20260811 22:31:38.475576 3726 webserver.cc:460] Webserver started at http://127.18.211.190:41873/ using document root <none> and password file <none>
I20260811 22:31:38.476246 3726 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:38.476346 3726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:38.476594 3726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:38.478792 3726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "a30a0fcbfdb348b78e068a2351b05e2a"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "48e65e900d0c8191ed3d6526d197a4f4"
server_key_iv: "0ea0514482dd2da2819201c411d51e92"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.479398 3726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "a30a0fcbfdb348b78e068a2351b05e2a"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "48e65e900d0c8191ed3d6526d197a4f4"
server_key_iv: "0ea0514482dd2da2819201c411d51e92"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.483340 3726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:38.487880 3756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:38.489157 3726 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 22:31:38.489341 3726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "a30a0fcbfdb348b78e068a2351b05e2a"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "48e65e900d0c8191ed3d6526d197a4f4"
server_key_iv: "0ea0514482dd2da2819201c411d51e92"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.489447 3726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:38.504053 3726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:38.504711 3726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:38.504861 3726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:38.512514 3726 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:43797
I20260811 22:31:38.512539 3844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:43797 every 8 connection(s)
I20260811 22:31:38.513578 3726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:38.515385 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 3726
I20260811 22:31:38.515525 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:38.515900 19278 external_mini_cluster.cc:1428] Setting key 62cc74ba2726abbbc7174f0cfbbd8ede
I20260811 22:31:38.517907 3845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:38.524080 3845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a: Bootstrap starting.
I20260811 22:31:38.525518 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:38.517496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.18.211.190:43797)
0811 22:31:38.518226 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:38.518242 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:38.518278 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:31:38.518952 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:38.518969 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:38.519357 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:38.519719 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:38.520145 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.521115 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.522042 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.522507 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.522700 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:38.523671 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:38.523676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:38.523695 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:38.524075 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:38.524128 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:38.524175 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:38.524333 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:38.524484 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 22:31:38.524744 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":152,"threads_started":1}
I20260811 22:31:38.526736 3845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:38.527719 3845 log.cc:824] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:38.529980 3845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a: No bootstrap required, opened a new log
I20260811 22:31:38.533808 3845 raft_consensus.cc:348] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43797 } }
I20260811 22:31:38.534076 3845 raft_consensus.cc:374] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:38.534178 3845 raft_consensus.cc:729] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30a0fcbfdb348b78e068a2351b05e2a, State: Initialized, Role: FOLLOWER
I20260811 22:31:38.534641 3845 consensus_queue.cc:260] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43797 } }
I20260811 22:31:38.534811 3845 raft_consensus.cc:388] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:38.534910 3845 raft_consensus.cc:482] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:38.535046 3845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:38.536024 3845 raft_consensus.cc:504] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43797 } }
I20260811 22:31:38.536412 3845 leader_election.cc:302] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a30a0fcbfdb348b78e068a2351b05e2a; no voters:
I20260811 22:31:38.536820 3845 leader_election.cc:288] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:38.536908 3861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:38.537228 3861 raft_consensus.cc:686] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [term 1 LEADER]: Becoming Leader. State: Replica: a30a0fcbfdb348b78e068a2351b05e2a, State: Running, Role: LEADER
I20260811 22:31:38.537549 3861 consensus_queue.cc:237] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43797 } }
I20260811 22:31:38.538204 3845 sys_catalog.cc:564] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:38.539347 3866 sys_catalog.cc:455] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a30a0fcbfdb348b78e068a2351b05e2a. Latest consensus state: current_term: 1 leader_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43797 } } }
I20260811 22:31:38.539482 3866 sys_catalog.cc:458] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:38.539353 3862 sys_catalog.cc:455] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30a0fcbfdb348b78e068a2351b05e2a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43797 } } }
I20260811 22:31:38.539795 3862 sys_catalog.cc:458] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:38.540035 3873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:38.543702 3873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:38.549582 3873 catalog_manager.cc:1347] Generated new cluster ID: d112dba65ba54d7f94f748861df2d572
I20260811 22:31:38.549675 3873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:38.591265 3873 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:38.592698 3873 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:38.602375 3873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a30a0fcbfdb348b78e068a2351b05e2a: Generated new TSK 0
I20260811 22:31:38.603204 3873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:31:38.614351 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43797
--builtin_ntp_servers=127.18.211.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 22:31:38.732556 3892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:38.732826 3892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:38.732865 3892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:38.736882 3892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:38.736941 3892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:38.736992 3892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:38.742035 3892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:38.743129 3892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:38.744516 3892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:38.754303 3906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:38.754277 3904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:38.754376 3892 server_base.cc:1034] running on GCE node
W20260811 22:31:38.754290 3910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:38.754920 3892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:38.755577 3892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:38.756775 3892 hybrid_clock.cc:648] HybridClock initialized: now 1786487498756759 us; error 29 us; skew 500 ppm
I20260811 22:31:38.758765 3892 webserver.cc:460] Webserver started at http://127.18.211.129:40005/ using document root <none> and password file <none>
I20260811 22:31:38.759363 3892 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:38.759457 3892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:38.759707 3892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:38.761897 3892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "46693921c9334f19aa8327edc660d19e"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "b6d44b9605fb270cca531639cd37e4b4"
server_key_iv: "abe48cc69e4726098448994f34a7acb7"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.762483 3892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "46693921c9334f19aa8327edc660d19e"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "b6d44b9605fb270cca531639cd37e4b4"
server_key_iv: "abe48cc69e4726098448994f34a7acb7"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.766252 3892 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:31:38.768920 3927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:38.770213 3892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:38.770354 3892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "46693921c9334f19aa8327edc660d19e"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "b6d44b9605fb270cca531639cd37e4b4"
server_key_iv: "abe48cc69e4726098448994f34a7acb7"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.770469 3892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:38.791471 3892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:38.792266 3892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:38.792479 3892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:38.793130 3892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:38.794174 3892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:38.794248 3892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:38.794320 3892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:38.794363 3892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:38.803174 3892 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:39081
I20260811 22:31:38.803207 4106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:39081 every 8 connection(s)
I20260811 22:31:38.804157 3892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:38.812654 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 3892
I20260811 22:31:38.812804 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:38.813165 19278 external_mini_cluster.cc:1428] Setting key 9cfe61bc2fd10d26e0793c13e71dce9e
I20260811 22:31:38.814126 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:38.806128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:51535 (local address 127.18.211.190:43797)
0811 22:31:38.806271 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:38.806275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:38.807339 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 22:31:38.808788 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:38.808792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:38.808859 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:38.808958 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:38.809928 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.810468 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.811633 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.811887 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.812004 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:38.813466 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:38.813473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:38.813479 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:38.813501 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:38.813530 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:38.813545 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:38.813651 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:38.813883 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 22:31:38.813976 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:38.815160 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:38.806512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43797 (local address 127.18.211.129:51535)
0811 22:31:38.807167 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:38.807218 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:38.808559 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:38.809149 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:38.809156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:38.809271 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:38.809777 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:38.809791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.810602 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.810610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:38.811499 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:38.811509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:38.812058 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:38.812063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:38.812239 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:38.813043 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:38.813046 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:38.813103 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:38.813196 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:38.813652 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:38.813663 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:38.813670 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:31:38.813873 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:31:38.814171 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":118,"threads_started":1}
I20260811 22:31:38.815521 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43797
--builtin_ntp_servers=127.18.211.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:31:38.816565 4108 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43797
I20260811 22:31:38.816809 4108 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:38.817443 4108 heartbeater.cc:511] Master 127.18.211.190:43797 requested a full tablet report, sending...
I20260811 22:31:38.819283 3787 ts_manager.cc:194] Registered new tserver with Master: 46693921c9334f19aa8327edc660d19e (127.18.211.129:39081)
I20260811 22:31:38.820886 3787 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.129:51535
W20260811 22:31:38.935403 4113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:38.935740 4113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:38.935822 4113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:38.940157 4113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:38.940270 4113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:38.940335 4113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:38.945122 4113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:38.946238 4113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:38.947593 4113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:38.957733 4126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:38.957757 4123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:38.957834 4113 server_base.cc:1034] running on GCE node
W20260811 22:31:38.957746 4122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:38.958433 4113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:38.959098 4113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:38.960284 4113 hybrid_clock.cc:648] HybridClock initialized: now 1786487498960266 us; error 28 us; skew 500 ppm
I20260811 22:31:38.962273 4113 webserver.cc:460] Webserver started at http://127.18.211.130:38095/ using document root <none> and password file <none>
I20260811 22:31:38.962868 4113 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:38.962983 4113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:38.963235 4113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:38.965366 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "7268f9b8babc424ab1b30a61521b6dac"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "268a7aafca667ff146e86f623298b0fc"
server_key_iv: "7a9174c970a8ac9406520e372b10e1fe"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.965946 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "7268f9b8babc424ab1b30a61521b6dac"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "268a7aafca667ff146e86f623298b0fc"
server_key_iv: "7a9174c970a8ac9406520e372b10e1fe"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.969648 4113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:38.972369 4133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:38.973906 4113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:38.974063 4113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "7268f9b8babc424ab1b30a61521b6dac"
format_stamp: "Formatted at 2026-08-11 22:31:38 on dist-test-slave-8hhm"
server_key: "268a7aafca667ff146e86f623298b0fc"
server_key_iv: "7a9174c970a8ac9406520e372b10e1fe"
server_key_version: "encryptionkey@0"
I20260811 22:31:38.974205 4113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:38.997030 4113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:38.997993 4113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:38.998176 4113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:38.998818 4113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:38.999873 4113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:38.999933 4113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:38.999981 4113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:39.000010 4113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.010661 4113 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:41065
I20260811 22:31:39.010685 4336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:41065 every 8 connection(s)
I20260811 22:31:39.011811 4113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:39.014106 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 4113
I20260811 22:31:39.014238 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:39.014559 19278 external_mini_cluster.cc:1428] Setting key 0ca05085e04c55db6cc2454818b29ad6
I20260811 22:31:39.016606 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43797
--builtin_ntp_servers=127.18.211.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:31:39.022024 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.013953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:58639 (local address 127.18.211.190:43797)
0811 22:31:39.014171 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:39.014176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:39.015316 (+ 1140us) server_negotiation.cc:407] Connection header received
0811 22:31:39.016720 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:39.016726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:39.016791 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:39.016893 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:39.017883 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.018544 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.019741 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.020045 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.020131 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:39.021499 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:39.021502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:39.021504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:39.021516 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:39.021535 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:39.021543 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:39.021602 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:39.021803 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 22:31:39.021888 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 22:31:39.023250 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.014421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43797 (local address 127.18.211.130:58639)
0811 22:31:39.015153 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:39.015207 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:39.016508 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:39.017075 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:39.017082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:39.017166 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:39.017734 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:39.017749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.018693 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.018696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:39.019617 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:39.019626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.020207 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.020214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:39.020456 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:39.021156 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:39.021161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:39.021251 (+ 90us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:39.021339 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:39.021636 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:39.021648 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:39.021658 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:31:39.021798 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 22:31:39.022130 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":134,"threads_started":1}
I20260811 22:31:39.024461 4337 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43797
I20260811 22:31:39.024727 4337 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:39.025326 4337 heartbeater.cc:511] Master 127.18.211.190:43797 requested a full tablet report, sending...
I20260811 22:31:39.026425 3787 ts_manager.cc:194] Registered new tserver with Master: 7268f9b8babc424ab1b30a61521b6dac (127.18.211.130:41065)
I20260811 22:31:39.027292 3787 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.130:58639
W20260811 22:31:39.139569 4342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:39.139873 4342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:39.139947 4342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:39.144040 4342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:39.144124 4342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:39.144222 4342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:39.149153 4342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:39.150319 4342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:39.151839 4342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:39.161736 4347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:39.161831 4352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:39.161743 4349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:39.162678 4342 server_base.cc:1034] running on GCE node
I20260811 22:31:39.163106 4342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:39.163800 4342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:39.165006 4342 hybrid_clock.cc:648] HybridClock initialized: now 1786487499164975 us; error 53 us; skew 500 ppm
I20260811 22:31:39.166990 4342 webserver.cc:460] Webserver started at http://127.18.211.131:39369/ using document root <none> and password file <none>
I20260811 22:31:39.167630 4342 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:39.167732 4342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:39.167945 4342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:39.170094 4342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "d3ad0d4262cf4d8bafa943ff426c01a6"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "a21f6b79628e38698705dd1e98a575e4"
server_key_iv: "73d5984acf253e335a2f05dc9f7125d4"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.170744 4342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "d3ad0d4262cf4d8bafa943ff426c01a6"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "a21f6b79628e38698705dd1e98a575e4"
server_key_iv: "73d5984acf253e335a2f05dc9f7125d4"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.175011 4342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:31:39.177764 4364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.179179 4342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:39.179332 4342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "d3ad0d4262cf4d8bafa943ff426c01a6"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "a21f6b79628e38698705dd1e98a575e4"
server_key_iv: "73d5984acf253e335a2f05dc9f7125d4"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.179450 4342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:39.203419 4342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:39.204100 4342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:39.204296 4342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:39.204906 4342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:39.205997 4342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:39.206074 4342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.206147 4342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:39.206197 4342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.215098 4342 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:43745
I20260811 22:31:39.215127 4587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:43745 every 8 connection(s)
I20260811 22:31:39.216001 4342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:39.218070 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 4342
I20260811 22:31:39.218176 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:39.218465 19278 external_mini_cluster.cc:1428] Setting key 8835415348a41243ad2ff734b28f5fce
I20260811 22:31:39.225452 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.217874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:42401 (local address 127.18.211.190:43797)
0811 22:31:39.218062 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:39.218067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:39.219377 (+ 1310us) server_negotiation.cc:407] Connection header received
0811 22:31:39.220944 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:39.220946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:39.220991 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:39.221057 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:39.221952 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.222461 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.223642 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.223857 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.223938 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:39.224985 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:39.224988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:39.224990 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:39.225001 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:39.225021 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:39.225029 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:39.225090 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:39.225287 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 22:31:39.225342 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:31:39.226305 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.218435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43797 (local address 127.18.211.131:42401)
0811 22:31:39.219222 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:39.219276 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:39.220755 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:39.221195 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:39.221201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:39.221342 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:39.221828 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:39.221840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.222584 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.222587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:39.223519 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:39.223528 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:39.224028 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:39.224032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:39.224181 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:39.224740 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:39.224742 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:39.224781 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:39.224841 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:39.225104 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:39.225111 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:39.225115 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:31:39.225272 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:31:39.225508 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":188,"threads_started":1}
I20260811 22:31:39.227530 4590 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43797
I20260811 22:31:39.227785 4590 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:39.228304 4590 heartbeater.cc:511] Master 127.18.211.190:43797 requested a full tablet report, sending...
I20260811 22:31:39.229398 3787 ts_manager.cc:194] Registered new tserver with Master: d3ad0d4262cf4d8bafa943ff426c01a6 (127.18.211.131:43745)
I20260811 22:31:39.229976 3787 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.131:42401
I20260811 22:31:39.232585 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:31:39.235514 3851 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:39.234515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35786 (local address 127.18.211.190:43797)
0811 22:31:39.234637 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:39.234641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:39.234853 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:31:39.234953 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:39.234955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:39.234992 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:39.235058 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:39.235371 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35786: BlockingRecv error: recv got EOF from 127.0.0.1:35786 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:31:39.236677 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 3892
I20260811 22:31:39.253665 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 4113
I20260811 22:31:39.264083 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 4342
I20260811 22:31:39.276907 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 3726
2026-08-11T22:31:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (986 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T22:31:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:39Z Disabled control of system clock
I20260811 22:31:39.314008 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34177
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:34177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:31:39.427920 4613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:39.428181 4613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:39.428259 4613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:39.432258 4613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:39.432314 4613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:39.432329 4613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:39.432345 4613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:39.432361 4613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:39.437107 4613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:34177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:39.438220 4613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:39.439637 4613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:39.448184 4627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:39.448161 4622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:39.448431 4613 server_base.cc:1034] running on GCE node
W20260811 22:31:39.448199 4621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:39.449019 4613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:39.450227 4613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:39.451417 4613 hybrid_clock.cc:648] HybridClock initialized: now 1786487499451404 us; error 25 us; skew 500 ppm
I20260811 22:31:39.453392 4613 webserver.cc:460] Webserver started at http://127.18.211.190:42907/ using document root <none> and password file <none>
I20260811 22:31:39.453991 4613 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:39.454098 4613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:39.454332 4613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:39.456427 4613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "7066b83b0a2e4134a869f2a00c1618d1"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "23ba3564f3eb74c9021a0fb516ca487f"
server_key_iv: "286100837d50c019612f58ac5e0a470e"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.457022 4613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "7066b83b0a2e4134a869f2a00c1618d1"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "23ba3564f3eb74c9021a0fb516ca487f"
server_key_iv: "286100837d50c019612f58ac5e0a470e"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.460831 4613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:31:39.463320 4635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.464561 4613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:39.464718 4613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "7066b83b0a2e4134a869f2a00c1618d1"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "23ba3564f3eb74c9021a0fb516ca487f"
server_key_iv: "286100837d50c019612f58ac5e0a470e"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.464835 4613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:39.469713 4613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:39.470345 4613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:39.470533 4613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:39.477475 4613 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:34177
I20260811 22:31:39.477480 4732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:34177 every 8 connection(s)
I20260811 22:31:39.478396 4613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:39.480024 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 4613
I20260811 22:31:39.480108 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:39.480392 19278 external_mini_cluster.cc:1428] Setting key 09901f4ed9c15ee32830259f3ce06255
I20260811 22:31:39.482211 4733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:39.486610 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.481823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47788 (local address 127.18.211.190:34177)
0811 22:31:39.482527 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:39.482546 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:39.482601 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:31:39.483447 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:39.483471 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:39.483922 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:39.484305 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:39.485032 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:39.485038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:39.485054 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:39.485065 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:39.485131 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:39.485157 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:39.485356 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:39.485439 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:31:39.485705 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
I20260811 22:31:39.489652 4733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1: Bootstrap starting.
I20260811 22:31:39.492298 4733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:39.493083 4733 log.cc:824] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:39.495041 4733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1: No bootstrap required, opened a new log
I20260811 22:31:39.498756 4733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7066b83b0a2e4134a869f2a00c1618d1" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34177 } }
I20260811 22:31:39.498974 4733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:39.499068 4733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7066b83b0a2e4134a869f2a00c1618d1, State: Initialized, Role: FOLLOWER
I20260811 22:31:39.499466 4733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7066b83b0a2e4134a869f2a00c1618d1" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34177 } }
I20260811 22:31:39.499612 4733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:39.499675 4733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:39.499781 4733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:39.500839 4733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7066b83b0a2e4134a869f2a00c1618d1" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34177 } }
I20260811 22:31:39.501242 4733 leader_election.cc:302] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7066b83b0a2e4134a869f2a00c1618d1; no voters:
I20260811 22:31:39.501652 4733 leader_election.cc:288] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:39.501781 4748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:39.502076 4748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [term 1 LEADER]: Becoming Leader. State: Replica: 7066b83b0a2e4134a869f2a00c1618d1, State: Running, Role: LEADER
I20260811 22:31:39.502424 4748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7066b83b0a2e4134a869f2a00c1618d1" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34177 } }
I20260811 22:31:39.502947 4733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:39.503984 4752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7066b83b0a2e4134a869f2a00c1618d1. Latest consensus state: current_term: 1 leader_uuid: "7066b83b0a2e4134a869f2a00c1618d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7066b83b0a2e4134a869f2a00c1618d1" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34177 } } }
I20260811 22:31:39.504487 4752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:39.504318 4750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7066b83b0a2e4134a869f2a00c1618d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7066b83b0a2e4134a869f2a00c1618d1" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34177 } } }
I20260811 22:31:39.504854 4750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:39.505478 4761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:39.508091 4761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:39.513662 4761 catalog_manager.cc:1347] Generated new cluster ID: 6521bf975371496684782c04a8428544
I20260811 22:31:39.513748 4761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:39.532958 4761 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:39.533346 4761 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:39.539064 4761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7066b83b0a2e4134a869f2a00c1618d1: Generated new TSK 0
I20260811 22:31:39.539760 4761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:31:39.551807 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34177
--builtin_ntp_servers=127.18.211.148:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:31:39.667981 4776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:39.668247 4776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:39.668286 4776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:39.672427 4776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:39.672484 4776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:39.672572 4776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:39.677296 4776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:39.678392 4776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:39.679818 4776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:39.689657 4791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:39.689718 4776 server_base.cc:1034] running on GCE node
W20260811 22:31:39.689667 4789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:39.689648 4794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:39.690279 4776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:39.690949 4776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:39.692158 4776 hybrid_clock.cc:648] HybridClock initialized: now 1786487499692126 us; error 42 us; skew 500 ppm
I20260811 22:31:39.694085 4776 webserver.cc:460] Webserver started at http://127.18.211.129:40515/ using document root <none> and password file <none>
I20260811 22:31:39.694684 4776 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:39.694778 4776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:39.695040 4776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:39.697139 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "ac984a37056a4f3586ea40af26f90804"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "a207bb71ce18cfd8566c5ffa90c5dbc7"
server_key_iv: "158fb242f58102c25c99e4850d159e70"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.697788 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "ac984a37056a4f3586ea40af26f90804"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "a207bb71ce18cfd8566c5ffa90c5dbc7"
server_key_iv: "158fb242f58102c25c99e4850d159e70"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.701660 4776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:31:39.704317 4805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.705689 4776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:31:39.705825 4776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "ac984a37056a4f3586ea40af26f90804"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "a207bb71ce18cfd8566c5ffa90c5dbc7"
server_key_iv: "158fb242f58102c25c99e4850d159e70"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.705943 4776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:39.710548 4776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:39.711158 4776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:39.711344 4776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:39.711899 4776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:39.712817 4776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:39.712890 4776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.712958 4776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:39.713021 4776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.721937 4776 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:40911
I20260811 22:31:39.721978 5016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:40911 every 8 connection(s)
I20260811 22:31:39.722829 4776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:39.728820 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.724711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:53395 (local address 127.18.211.190:34177)
0811 22:31:39.724871 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:39.724874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:39.725971 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 22:31:39.727241 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:39.727244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:39.727281 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:39.727354 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:39.728365 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:39.728368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:39.728371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:39.728381 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:39.728400 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:39.728407 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:39.728480 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:39.728622 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 22:31:39.728657 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:31:39.729077 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 4776
I20260811 22:31:39.729190 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:39.729503 19278 external_mini_cluster.cc:1428] Setting key 882d915be432e5f27c4675d0baeff1ed
I20260811 22:31:39.729784 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.725094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:34177 (local address 127.18.211.129:53395)
0811 22:31:39.725815 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:39.725855 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:39.727046 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:39.727532 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:39.727538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:39.727622 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:39.728118 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:39.728120 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:39.728157 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:39.728220 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:39.728491 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:39.728497 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:39.728501 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:31:39.728636 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:31:39.728838 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":178,"threads_started":1}
I20260811 22:31:39.731092 5017 heartbeater.cc:347] Connected to a master server at 127.18.211.190:34177
I20260811 22:31:39.731333 5017 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:39.731581 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34177
--builtin_ntp_servers=127.18.211.148:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:31:39.731815 5017 heartbeater.cc:511] Master 127.18.211.190:34177 requested a full tablet report, sending...
I20260811 22:31:39.733434 4672 ts_manager.cc:194] Registered new tserver with Master: ac984a37056a4f3586ea40af26f90804 (127.18.211.129:40911)
W20260811 22:31:39.849323 5024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:39.849571 5024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:39.849610 5024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:39.853487 5024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:39.853538 5024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:39.853591 5024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:39.858264 5024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:39.859277 5024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:39.860605 5024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:39.870625 5034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:39.870697 5033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:39.870672 5038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:39.870847 5024 server_base.cc:1034] running on GCE node
I20260811 22:31:39.871361 5024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:39.871994 5024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:39.873167 5024 hybrid_clock.cc:648] HybridClock initialized: now 1786487499873132 us; error 48 us; skew 500 ppm
I20260811 22:31:39.875074 5024 webserver.cc:460] Webserver started at http://127.18.211.130:43497/ using document root <none> and password file <none>
I20260811 22:31:39.875627 5024 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:39.875687 5024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:39.875847 5024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:39.877969 5024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "d52dbd35a5de4506851579798c133e9e"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "1ed9825efa7986e303204b963e8cfafe"
server_key_iv: "f5f355dfd8dda5eb8fca584aec50dbcb"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.878489 5024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "d52dbd35a5de4506851579798c133e9e"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "1ed9825efa7986e303204b963e8cfafe"
server_key_iv: "f5f355dfd8dda5eb8fca584aec50dbcb"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.882225 5024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:31:39.884707 5047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.886125 5024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:39.886258 5024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "d52dbd35a5de4506851579798c133e9e"
format_stamp: "Formatted at 2026-08-11 22:31:39 on dist-test-slave-8hhm"
server_key: "1ed9825efa7986e303204b963e8cfafe"
server_key_iv: "f5f355dfd8dda5eb8fca584aec50dbcb"
server_key_version: "encryptionkey@0"
I20260811 22:31:39.886370 5024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:39.890875 5024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:39.891449 5024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:39.891629 5024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:39.892164 5024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:39.893038 5024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:39.893107 5024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.893177 5024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:39.893245 5024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:39.901778 5024 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:42781
I20260811 22:31:39.901839 5242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:42781 every 8 connection(s)
I20260811 22:31:39.902669 5024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:39.909085 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.904658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:52397 (local address 127.18.211.190:34177)
0811 22:31:39.904790 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:39.904812 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:39.905948 (+ 1136us) server_negotiation.cc:407] Connection header received
0811 22:31:39.907287 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:39.907291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:39.907336 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:39.907421 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:39.908588 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:39.908592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:39.908595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:39.908608 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:39.908631 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:39.908648 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:39.908753 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:39.908902 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 22:31:39.908949 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:31:39.909541 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 5024
I20260811 22:31:39.909663 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:39.910009 19278 external_mini_cluster.cc:1428] Setting key 34f3a874d053acc9290a61bc14a6d0d4
I20260811 22:31:39.910202 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:39.905063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:34177 (local address 127.18.211.130:52397)
0811 22:31:39.905806 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:39.905846 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:39.907070 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:39.907588 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:39.907595 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:39.907687 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:39.908321 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:39.908322 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:39.908364 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:39.908432 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:39.908736 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:39.908746 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:39.908752 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:31:39.908934 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 22:31:39.909174 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":128,"threads_started":1}
I20260811 22:31:39.911273 5243 heartbeater.cc:347] Connected to a master server at 127.18.211.190:34177
I20260811 22:31:39.911521 5243 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:39.912010 5243 heartbeater.cc:511] Master 127.18.211.190:34177 requested a full tablet report, sending...
I20260811 22:31:39.912410 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34177
--builtin_ntp_servers=127.18.211.148:38725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:31:39.913928 4671 ts_manager.cc:194] Registered new tserver with Master: d52dbd35a5de4506851579798c133e9e (127.18.211.130:42781)
W20260811 22:31:40.028203 5246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:40.028448 5246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:40.028486 5246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:40.032413 5246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:40.032466 5246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:40.032518 5246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:40.037148 5246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:40.038193 5246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:40.039525 5246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:40.049063 5262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:40.049124 5252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:40.049099 5256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:40.049340 5246 server_base.cc:1034] running on GCE node
I20260811 22:31:40.049779 5246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:40.050405 5246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:40.051587 5246 hybrid_clock.cc:648] HybridClock initialized: now 1786487500051549 us; error 51 us; skew 500 ppm
I20260811 22:31:40.053478 5246 webserver.cc:460] Webserver started at http://127.18.211.131:37887/ using document root <none> and password file <none>
I20260811 22:31:40.054059 5246 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:40.054168 5246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:40.054404 5246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:40.056411 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "a302c73f44ed4c52ae36a07798147792"
format_stamp: "Formatted at 2026-08-11 22:31:40 on dist-test-slave-8hhm"
server_key: "61a6e832c7ce2d460afbd6d855257bdd"
server_key_iv: "a6a169c787ef16c55ac93e72d3f25d7b"
server_key_version: "encryptionkey@0"
I20260811 22:31:40.056981 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "a302c73f44ed4c52ae36a07798147792"
format_stamp: "Formatted at 2026-08-11 22:31:40 on dist-test-slave-8hhm"
server_key: "61a6e832c7ce2d460afbd6d855257bdd"
server_key_iv: "a6a169c787ef16c55ac93e72d3f25d7b"
server_key_version: "encryptionkey@0"
I20260811 22:31:40.060659 5246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:31:40.063211 5272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:40.064354 5246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:40.064487 5246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "a302c73f44ed4c52ae36a07798147792"
format_stamp: "Formatted at 2026-08-11 22:31:40 on dist-test-slave-8hhm"
server_key: "61a6e832c7ce2d460afbd6d855257bdd"
server_key_iv: "a6a169c787ef16c55ac93e72d3f25d7b"
server_key_version: "encryptionkey@0"
I20260811 22:31:40.064602 5246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:40.069175 5246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:40.069794 5246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:40.069983 5246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:40.070520 5246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:40.071419 5246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:40.071491 5246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:40.071555 5246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:40.071609 5246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:40.080711 5246 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:45571
I20260811 22:31:40.080740 5483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:45571 every 8 connection(s)
I20260811 22:31:40.081668 5246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:40.087752 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:40.083424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:46945 (local address 127.18.211.190:34177)
0811 22:31:40.083558 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:40.083562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:40.084544 (+ 982us) server_negotiation.cc:407] Connection header received
0811 22:31:40.085971 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:40.085975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:40.086023 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:40.086130 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:40.087261 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:40.087264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:40.087267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:31:40.087280 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:40.087311 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:31:40.087319 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:40.087418 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:40.087567 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 22:31:40.087616 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:31:40.088580 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:40.083768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:34177 (local address 127.18.211.131:46945)
0811 22:31:40.084394 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:40.084432 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:40.085758 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:40.086301 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:40.086307 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:40.086393 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:40.086989 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:31:40.086991 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:40.087031 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:31:40.087098 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:40.087402 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:40.087410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:40.087416 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:31:40.087598 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 22:31:40.087839 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":112,"threads_started":1}
I20260811 22:31:40.089749 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 5246
I20260811 22:31:40.089866 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:40.089895 5485 heartbeater.cc:347] Connected to a master server at 127.18.211.190:34177
I20260811 22:31:40.090196 5485 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:40.090297 19278 external_mini_cluster.cc:1428] Setting key 4b8cc218ede4076c20d1fcf27f0f51f7
I20260811 22:31:40.090689 5485 heartbeater.cc:511] Master 127.18.211.190:34177 requested a full tablet report, sending...
I20260811 22:31:40.092005 4671 ts_manager.cc:194] Registered new tserver with Master: a302c73f44ed4c52ae36a07798147792 (127.18.211.131:45571)
I20260811 22:31:40.104146 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:31:40.107106 4739 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:31:40.106140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.18.211.190:34177)
0811 22:31:40.106272 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:40.106278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:40.106445 (+ 167us) server_negotiation.cc:407] Connection header received
0811 22:31:40.106602 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:40.106605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:40.106651 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:40.106743 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:40.106980 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47804: BlockingRecv error: recv got EOF from 127.0.0.1:47804 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:31:40.107245 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 4776
I20260811 22:31:40.130033 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 5024
I20260811 22:31:40.148833 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 5246
I20260811 22:31:40.164388 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 4613
2026-08-11T22:31:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (887 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:31:40 dist-test-slave-8hhm krb5kdc[5499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:31:40 dist-test-slave-8hhm krb5kdc[5499](info): set up 2 sockets
Aug 11 22:31:40 dist-test-slave-8hhm krb5kdc[5499](info): commencing operation
krb5kdc: starting...
W20260811 22:31:41.468869 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:41 dist-test-slave-8hhm krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:31:41.601861 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45863
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:41.718737 5526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:41.719058 5526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:41.719141 5526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:41.723227 5526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:31:41.723317 5526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:41.723353 5526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:41.723392 5526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:31:41.723425 5526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:31:41.728677 5526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:41.729851 5526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:41.730902 5526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:41.739624 5533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:41.739674 5536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:41.739692 5546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:41.740024 5526 server_base.cc:1034] running on GCE node
I20260811 22:31:41.740559 5526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:41.741729 5526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:41.742899 5526 hybrid_clock.cc:648] HybridClock initialized: now 1786487501742865 us; error 45 us; skew 500 ppm
Aug 11 22:31:41 dist-test-slave-8hhm krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487501, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:41.745822 5526 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:31:41.746640 5526 webserver.cc:460] Webserver started at http://127.18.211.190:42565/ using document root <none> and password file <none>
I20260811 22:31:41.747157 5526 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:41.747228 5526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:41.747390 5526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:41.749504 5526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "b8da7b642c544d019f745a72a3130ea8"
format_stamp: "Formatted at 2026-08-11 22:31:41 on dist-test-slave-8hhm"
server_key: "7da3f4714a7dcf4a007d5081708b1ed1"
server_key_iv: "5bd17c48fefb142c38aef74ed1af0c05"
server_key_version: "encryptionkey@0"
I20260811 22:31:41.750068 5526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "b8da7b642c544d019f745a72a3130ea8"
format_stamp: "Formatted at 2026-08-11 22:31:41 on dist-test-slave-8hhm"
server_key: "7da3f4714a7dcf4a007d5081708b1ed1"
server_key_iv: "5bd17c48fefb142c38aef74ed1af0c05"
server_key_version: "encryptionkey@0"
I20260811 22:31:41.753710 5526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:31:41.755991 5556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:41.757200 5526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:41.757376 5526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "b8da7b642c544d019f745a72a3130ea8"
format_stamp: "Formatted at 2026-08-11 22:31:41 on dist-test-slave-8hhm"
server_key: "7da3f4714a7dcf4a007d5081708b1ed1"
server_key_iv: "5bd17c48fefb142c38aef74ed1af0c05"
server_key_version: "encryptionkey@0"
I20260811 22:31:41.757468 5526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:41.787997 5526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:41.791540 5526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:41.791719 5526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:41.799373 5526 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45863
I20260811 22:31:41.799461 5644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45863 every 8 connection(s)
I20260811 22:31:41.800468 5526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:31:41.803359 5645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:41.809160 5645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8: Bootstrap starting.
I20260811 22:31:41.809818 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 5526
I20260811 22:31:41.809926 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:31:41.810218 19278 external_mini_cluster.cc:1428] Setting key 5789de5b6057e5602a577aab5aa134fb
I20260811 22:31:41.812053 5645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:41.812958 5645 log.cc:824] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:41.815183 5645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8: No bootstrap required, opened a new log
I20260811 22:31:41.819139 5645 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da7b642c544d019f745a72a3130ea8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45863 } }
I20260811 22:31:41.819389 5645 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:41.819476 5645 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8da7b642c544d019f745a72a3130ea8, State: Initialized, Role: FOLLOWER
Aug 11 22:31:41 dist-test-slave-8hhm krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:41.819993 5645 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da7b642c544d019f745a72a3130ea8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45863 } }
I20260811 22:31:41.820153 5645 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:31:41.820243 5645 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:31:41.820351 5645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:41.821246 5645 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da7b642c544d019f745a72a3130ea8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45863 } }
I20260811 22:31:41.821619 5645 leader_election.cc:302] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8da7b642c544d019f745a72a3130ea8; no voters:
I20260811 22:31:41.822005 5645 leader_election.cc:288] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:31:41.822080 5654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:41.822322 5654 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [term 1 LEADER]: Becoming Leader. State: Replica: b8da7b642c544d019f745a72a3130ea8, State: Running, Role: LEADER
I20260811 22:31:41.822685 5654 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da7b642c544d019f745a72a3130ea8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45863 } }
I20260811 22:31:41.823319 5645 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:31:41.824497 5655 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8da7b642c544d019f745a72a3130ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da7b642c544d019f745a72a3130ea8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45863 } } }
I20260811 22:31:41.824637 5655 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:41.824942 5656 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8da7b642c544d019f745a72a3130ea8. Latest consensus state: current_term: 1 leader_uuid: "b8da7b642c544d019f745a72a3130ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8da7b642c544d019f745a72a3130ea8" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45863 } } }
I20260811 22:31:41.825098 5656 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:31:41.825433 5668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:31:41.826673 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:41.811595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57120 (local address 127.18.211.190:45863)
0811 22:31:41.812082 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:41.812103 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:41.812151 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:31:41.813019 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:41.813044 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:41.813501 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:41.813856 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:41.814539 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:41.815999 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:41.816857 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:41.817597 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:41.817785 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:41.820423 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:41.820428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:41.820441 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:41.820491 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:41.822594 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:41.823095 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:41.823099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:41.823104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:41.823219 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:41.823546 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:41.823550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:41.823552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:41.824111 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:41.824998 (+ 887us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:41.825161 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 22:31:41.825515 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":110,"threads_started":1}
I20260811 22:31:41.829804 5668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:31:41.835551 5668 catalog_manager.cc:1347] Generated new cluster ID: fd15f099071d4db4b100bef426e50a8c
I20260811 22:31:41.835675 5668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:31:41.851619 5668 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:31:41.852556 5668 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:31:41.868780 5668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8: Generated new TSK 0
I20260811 22:31:41.869653 5668 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:31:41.900199 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45863
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:42.013844 5692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:42.014101 5692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:42.014166 5692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:42.018208 5692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:42.018265 5692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:42.018352 5692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:31:42.023655 5692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:42.024778 5692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:42.025933 5692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:42.035842 5702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:42.035881 5705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:42.035864 5692 server_base.cc:1034] running on GCE node
W20260811 22:31:42.035903 5703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:42.036598 5692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:42.037154 5692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:42.038378 5692 hybrid_clock.cc:648] HybridClock initialized: now 1786487502038349 us; error 41 us; skew 500 ppm
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:42.041268 5692 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:31:42.042115 5692 webserver.cc:460] Webserver started at http://127.18.211.129:35623/ using document root <none> and password file <none>
I20260811 22:31:42.042644 5692 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:42.042707 5692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:42.042968 5692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:42.045362 5692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "4a142128d78e4e7bba509f1fcc4cd10d"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "754b93de2b4328ced8e0d362b1ac966e"
server_key_iv: "ab6af2b608bd758c1cc8adfba878e7c8"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.045897 5692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "4a142128d78e4e7bba509f1fcc4cd10d"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "754b93de2b4328ced8e0d362b1ac966e"
server_key_iv: "ab6af2b608bd758c1cc8adfba878e7c8"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.049495 5692 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:31:42.051883 5719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.053117 5692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:31:42.053306 5692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "4a142128d78e4e7bba509f1fcc4cd10d"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "754b93de2b4328ced8e0d362b1ac966e"
server_key_iv: "ab6af2b608bd758c1cc8adfba878e7c8"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.053404 5692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:42.076228 5692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:42.079712 5692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:42.079872 5692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:42.080421 5692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:42.081296 5692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:42.081354 5692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.081398 5692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:42.081413 5692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.090581 5692 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:35419
I20260811 22:31:42.090611 5937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:35419 every 8 connection(s)
I20260811 22:31:42.091460 5692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:31:42.097177 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 5692
I20260811 22:31:42.097319 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:31:42.097625 19278 external_mini_cluster.cc:1428] Setting key 5f61b9f4016902e4f2caf9489b86bc44
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:42.106161 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.093409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:56957 (local address 127.18.211.190:45863)
0811 22:31:42.093544 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.093548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.094360 (+ 812us) server_negotiation.cc:407] Connection header received
0811 22:31:42.095680 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.095683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.095731 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.095806 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:42.096928 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.097737 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.099359 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.099714 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.099832 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.102224 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:42.102228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:42.102231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:42.102257 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:42.103785 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.104322 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.104330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.104332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.104411 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.104863 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.104868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.104870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.105051 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:42.105179 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.105870 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.106001 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:31:42.106878 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.093726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45863 (local address 127.18.211.129:56957)
0811 22:31:42.094191 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.094235 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.095470 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.095939 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.095945 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.096290 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:42.096762 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.096775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.097857 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.097860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.099238 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.099249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.099852 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.099855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.099988 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.100585 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:42.100600 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:42.102055 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:42.103919 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:42.103924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:42.103938 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:42.104208 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:42.104506 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:42.104510 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:42.104512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:42.104750 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:42.105178 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:42.105196 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:42.105488 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.105691 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.106017 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:31:42.108142 5938 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45863
I20260811 22:31:42.108419 5938 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:42.108911 5938 heartbeater.cc:511] Master 127.18.211.190:45863 requested a full tablet report, sending...
I20260811 22:31:42.110589 5586 ts_manager.cc:194] Registered new tserver with Master: 4a142128d78e4e7bba509f1fcc4cd10d (127.18.211.129:35419)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:31:42.111826 5586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:56957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:31:42.128077 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45863
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:42.245229 5948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:42.245481 5948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:42.245524 5948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:42.249424 5948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:42.249475 5948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:42.249528 5948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:31:42.254544 5948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:42.255581 5948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:42.256685 5948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:42.266476 5955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:42.266523 5954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:42.266685 5948 server_base.cc:1034] running on GCE node
W20260811 22:31:42.266832 5959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:42.267261 5948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:42.267940 5948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:42.269146 5948 hybrid_clock.cc:648] HybridClock initialized: now 1786487502269135 us; error 29 us; skew 500 ppm
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:42.271891 5948 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:31:42.272855 5948 webserver.cc:460] Webserver started at http://127.18.211.130:42709/ using document root <none> and password file <none>
I20260811 22:31:42.273497 5948 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:42.273590 5948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:42.273831 5948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:42.275866 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "0c4263e82c1744fc9017e24599ed7eb5"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "7cc903fac8111afc08644fb42599f9f6"
server_key_iv: "0adcc7169e0a04d66b3e3026450173d7"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.276448 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "0c4263e82c1744fc9017e24599ed7eb5"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "7cc903fac8111afc08644fb42599f9f6"
server_key_iv: "0adcc7169e0a04d66b3e3026450173d7"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.280206 5948 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 22:31:42.282645 5968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.283831 5948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:31:42.283968 5948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "0c4263e82c1744fc9017e24599ed7eb5"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "7cc903fac8111afc08644fb42599f9f6"
server_key_iv: "0adcc7169e0a04d66b3e3026450173d7"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.284080 5948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:42.305743 5948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:42.309396 5948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:42.309621 5948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:42.310271 5948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:42.311153 5948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:42.311226 5948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.311295 5948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:42.311347 5948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.320433 5948 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:41915
I20260811 22:31:42.320458 6189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:41915 every 8 connection(s)
I20260811 22:31:42.321426 5948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:31:42.325958 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 5948
I20260811 22:31:42.326054 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:31:42.326427 19278 external_mini_cluster.cc:1428] Setting key 56e329d0e23b30d6224e659e0fb3d3dc
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:42.336640 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.323354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:56825 (local address 127.18.211.190:45863)
0811 22:31:42.323505 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.323510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.324482 (+ 972us) server_negotiation.cc:407] Connection header received
0811 22:31:42.326266 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.326271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.326334 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.326439 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:42.327844 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.328603 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.329883 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.330182 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.330267 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.332984 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:42.332987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:42.332989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:42.333026 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:42.334530 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.335067 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.335070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.335073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.335159 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.335508 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.335513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.335515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.335706 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:42.335818 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.336328 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.336456 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:31:42.337530 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.323769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45863 (local address 127.18.211.130:56825)
0811 22:31:42.324305 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.324358 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.325988 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.326648 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.326655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.327133 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:42.327671 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.327690 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.328733 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.328736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.329706 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.329716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.330342 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.330347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.330580 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.331263 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:42.331287 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:42.332811 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:42.334664 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:42.334670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:42.334685 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:42.334952 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:42.335260 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:42.335263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:42.335265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:42.335409 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:42.335817 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:42.335824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:42.336076 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.336271 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.336587 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":144,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:31:42.338798 6190 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45863
I20260811 22:31:42.339061 6190 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:42.339566 6190 heartbeater.cc:511] Master 127.18.211.190:45863 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:31:42.340691 5586 ts_manager.cc:194] Registered new tserver with Master: 0c4263e82c1744fc9017e24599ed7eb5 (127.18.211.130:41915)
I20260811 22:31:42.341365 5586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:56825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:31:42.356848 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45863
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:31:42.472919 6200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:31:42.473255 6200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:31:42.473340 6200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:31:42.477437 6200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:31:42.477489 6200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:31:42.477545 6200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:31:42.482774 6200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:31:42.483810 6200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:31:42.484848 6200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:31:42.495044 6215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:42.495127 6200 server_base.cc:1034] running on GCE node
W20260811 22:31:42.495059 6217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:31:42.495069 6220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:31:42.495709 6200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:31:42.496378 6200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:31:42.497561 6200 hybrid_clock.cc:648] HybridClock initialized: now 1786487502497547 us; error 25 us; skew 500 ppm
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:31:42.500347 6200 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:31:42.501304 6200 webserver.cc:460] Webserver started at http://127.18.211.131:46127/ using document root <none> and password file <none>
I20260811 22:31:42.501935 6200 fs_manager.cc:359] Metadata directory not provided
I20260811 22:31:42.502031 6200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:31:42.502271 6200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:31:42.504328 6200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "22676fbd58a34ee08a80b98f9f169f51"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "c26cf62b47ddbb6178dfbca1d02487bd"
server_key_iv: "5f4e3c7802f15a0c42e6922338f09529"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.504997 6200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "22676fbd58a34ee08a80b98f9f169f51"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "c26cf62b47ddbb6178dfbca1d02487bd"
server_key_iv: "5f4e3c7802f15a0c42e6922338f09529"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.509698 6200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:31:42.512177 6235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.513504 6200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:31:42.513636 6200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "22676fbd58a34ee08a80b98f9f169f51"
format_stamp: "Formatted at 2026-08-11 22:31:42 on dist-test-slave-8hhm"
server_key: "c26cf62b47ddbb6178dfbca1d02487bd"
server_key_iv: "5f4e3c7802f15a0c42e6922338f09529"
server_key_version: "encryptionkey@0"
I20260811 22:31:42.513758 6200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:31:42.527483 6200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:31:42.530455 6200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:31:42.530666 6200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:31:42.531256 6200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:31:42.532094 6200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:31:42.532167 6200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.532228 6200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:31:42.532279 6200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:31:42.541277 6200 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:33239
I20260811 22:31:42.541307 6442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:33239 every 8 connection(s)
I20260811 22:31:42.542126 6200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:31:42.544260 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 6200
I20260811 22:31:42.544392 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:31:42.544824 19278 external_mini_cluster.cc:1428] Setting key e846dc016df7914b52f5968bfa0ead97
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:31:42.555702 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.543893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:35467 (local address 127.18.211.190:45863)
0811 22:31:42.544081 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.544099 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.545055 (+ 956us) server_negotiation.cc:407] Connection header received
0811 22:31:42.546928 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.546932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.546986 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.547075 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:42.548235 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.548748 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.549924 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.550128 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.550237 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.552532 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:42.552535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:42.552537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:42.552559 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:42.554014 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.554461 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.554463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.554464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.554537 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.554824 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.554826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.554827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.554985 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:42.555060 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.555492 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.555581 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:42.556516 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.544300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45863 (local address 127.18.211.131:35467)
0811 22:31:42.544878 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.544939 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.546710 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.547260 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.547275 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.547621 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:31:42.548103 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.548116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.548856 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.548858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.549815 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.549825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.550272 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.550275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.550415 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.550911 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:31:42.550926 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:31:42.552382 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:31:42.554113 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:42.554117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:42.554128 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:42.554374 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:42.554616 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:31:42.554618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:31:42.554620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:31:42.554747 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:31:42.555085 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:31:42.555090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:31:42.555297 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.555444 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.555673 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":170,"threads_started":1}
I20260811 22:31:42.557765 6443 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45863
I20260811 22:31:42.558043 6443 heartbeater.cc:464] Registering TS with master...
I20260811 22:31:42.558573 6443 heartbeater.cc:511] Master 127.18.211.190:45863 requested a full tablet report, sending...
I20260811 22:31:42.559580 5586 ts_manager.cc:194] Registered new tserver with Master: 22676fbd58a34ee08a80b98f9f169f51 (127.18.211.131:33239)
I20260811 22:31:42.560132 5586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:35467
I20260811 22:31:42.570557 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:31:42.580102 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.572554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57130 (local address 127.18.211.190:45863)
0811 22:31:42.572723 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.572729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.572960 (+ 231us) server_negotiation.cc:407] Connection header received
0811 22:31:42.573134 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.573136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.573175 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.573305 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:31:42.574036 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.574521 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.575330 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.575533 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.575636 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.577049 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:31:42.577053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:31:42.577055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:31:42.577077 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:31:42.578425 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.578843 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.578847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.578849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.578929 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:31:42.579239 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:31:42.579242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:31:42.579244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:31:42.579445 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:31:42.579605 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.579829 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.579934 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:31:42.582739 5586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:31:42.585793 5586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:31:42.600076 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.594380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.18.211.131:33239)
0811 22:31:42.594917 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.594930 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.594955 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:31:42.595053 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.595062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.595280 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.595456 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:42.595968 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.596911 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.598572 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.599423 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.599528 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.599659 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.599834 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.599927 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":181,"threads_started":1}
I20260811 22:31:42.600064 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.594240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:33239 (local address 127.0.0.1:42186)
0811 22:31:42.594571 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.594626 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.594761 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.595654 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.595660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.595682 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:42.595822 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.595830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.597054 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.597057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.598432 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.598441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.599521 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.599524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.599635 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.599671 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.599804 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.599923 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
I20260811 22:31:42.600705 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.594953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:35419 (local address 127.0.0.1:59008)
0811 22:31:42.595554 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.595567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.595665 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.596760 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.596763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.596782 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:42.596874 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.596881 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.597998 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.598003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.599081 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.599091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.600276 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.600280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.600385 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.600407 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.600494 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.600572 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":99,"threads_started":1}
I20260811 22:31:42.600765 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.595113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.18.211.129:35419)
0811 22:31:42.596182 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.596197 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.596216 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:31:42.596279 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.596284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.596442 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.596573 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:42.597014 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.597846 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.599209 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.599970 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.600096 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.600213 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.600538 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.600630 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":125,"threads_started":1}
I20260811 22:31:42.601317 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.595298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41915 (local address 127.0.0.1:42036)
0811 22:31:42.595904 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.595919 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.595997 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.596478 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.596481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.596500 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:42.596612 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.596617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.598141 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.598145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.599313 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.599325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.600721 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.600727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.600991 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.601010 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.601113 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.601182 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":92,"threads_started":1}
I20260811 22:31:42.601672 6472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.595487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.18.211.130:41915)
0811 22:31:42.595958 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.595970 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.596006 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:31:42.596133 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.596139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.596321 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.596475 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:42.596746 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.597956 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.599455 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.600428 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.600540 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.600657 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.601150 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.601509 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":145,"threads_started":1}
I20260811 22:31:42.603437 6318 tablet_service.cc:1467] Processing CreateTablet for tablet 24e32a55f5de4ff0a8c151f4a0a5b537 (DEFAULT_TABLE table=test-table [id=3e8bbe2de0f34b2ea38ba30aef253741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:42.603806 5814 tablet_service.cc:1467] Processing CreateTablet for tablet 24e32a55f5de4ff0a8c151f4a0a5b537 (DEFAULT_TABLE table=test-table [id=3e8bbe2de0f34b2ea38ba30aef253741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:42.604372 6068 tablet_service.cc:1467] Processing CreateTablet for tablet 24e32a55f5de4ff0a8c151f4a0a5b537 (DEFAULT_TABLE table=test-table [id=3e8bbe2de0f34b2ea38ba30aef253741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:31:42.604722 6318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e32a55f5de4ff0a8c151f4a0a5b537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:42.604820 5814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e32a55f5de4ff0a8c151f4a0a5b537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:42.605337 6068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e32a55f5de4ff0a8c151f4a0a5b537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:31:42.611852 6474 tablet_bootstrap.cc:492] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Bootstrap starting.
I20260811 22:31:42.612736 6475 tablet_bootstrap.cc:492] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d: Bootstrap starting.
I20260811 22:31:42.613561 6477 tablet_bootstrap.cc:492] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Bootstrap starting.
I20260811 22:31:42.614534 6474 tablet_bootstrap.cc:654] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:42.614811 6475 tablet_bootstrap.cc:654] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:42.615332 6474 log.cc:824] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:42.615541 6477 tablet_bootstrap.cc:654] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Neither blocks nor log segments found. Creating new log.
I20260811 22:31:42.615648 6475 log.cc:824] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:42.616757 6477 log.cc:824] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Log is configured to *not* fsync() on all Append() calls
I20260811 22:31:42.617487 6474 tablet_bootstrap.cc:492] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: No bootstrap required, opened a new log
I20260811 22:31:42.617697 6474 ts_tablet_manager.cc:1397] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:31:42.618126 6475 tablet_bootstrap.cc:492] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d: No bootstrap required, opened a new log
I20260811 22:31:42.618322 6475 ts_tablet_manager.cc:1397] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:31:42.618753 6477 tablet_bootstrap.cc:492] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: No bootstrap required, opened a new log
I20260811 22:31:42.618937 6477 ts_tablet_manager.cc:1397] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:31:42.620810 6474 raft_consensus.cc:348] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.621063 6474 raft_consensus.cc:374] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:42.621155 6474 raft_consensus.cc:729] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4263e82c1744fc9017e24599ed7eb5, State: Initialized, Role: FOLLOWER
I20260811 22:31:42.621654 6474 consensus_queue.cc:260] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.622306 6190 heartbeater.cc:503] Master 127.18.211.190:45863 was elected leader, sending a full tablet report...
I20260811 22:31:42.622615 6474 ts_tablet_manager.cc:1428] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 22:31:42.622802 6475 raft_consensus.cc:348] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.623060 6475 raft_consensus.cc:374] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:42.623143 6475 raft_consensus.cc:729] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a142128d78e4e7bba509f1fcc4cd10d, State: Initialized, Role: FOLLOWER
I20260811 22:31:42.623664 6477 raft_consensus.cc:348] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.623930 6475 consensus_queue.cc:260] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.624002 6477 raft_consensus.cc:374] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:31:42.624089 6477 raft_consensus.cc:729] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22676fbd58a34ee08a80b98f9f169f51, State: Initialized, Role: FOLLOWER
I20260811 22:31:42.624644 6477 consensus_queue.cc:260] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.624943 6475 ts_tablet_manager.cc:1428] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:31:42.625408 6443 heartbeater.cc:503] Master 127.18.211.190:45863 was elected leader, sending a full tablet report...
I20260811 22:31:42.625679 5938 heartbeater.cc:503] Master 127.18.211.190:45863 was elected leader, sending a full tablet report...
I20260811 22:31:42.625697 6477 ts_tablet_manager.cc:1428] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:31:42.646898 6489 raft_consensus.cc:482] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:31:42.647118 6489 raft_consensus.cc:504] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.648262 6489 leader_election.cc:288] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22676fbd58a34ee08a80b98f9f169f51 (127.18.211.131:33239), 4a142128d78e4e7bba509f1fcc4cd10d (127.18.211.129:35419)
I20260811 22:31:42.652921 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.648553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:33239 (local address 127.18.211.130:43231)
0811 22:31:42.648703 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.648722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.648823 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.649182 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.649185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.649207 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:42.649365 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.649374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.650342 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.650344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.651579 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.651590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.652431 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.652433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.652547 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.652606 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.652685 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.652768 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":24}
I20260811 22:31:42.652936 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.648649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:43231 (local address 127.18.211.131:33239)
0811 22:31:42.648796 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.648802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.648817 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:31:42.648945 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.648948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.648997 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.649086 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:42.649475 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.650231 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.651718 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.652310 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.652437 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.652502 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.652723 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.652817 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:31:42.652977 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.648980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:45313 (local address 127.18.211.129:35419)
0811 22:31:42.649123 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.649128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.649310 (+ 182us) server_negotiation.cc:407] Connection header received
0811 22:31:42.649411 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.649413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.649480 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.649558 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:31:42.649801 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.650465 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.651841 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.652448 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.652556 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.652601 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.652801 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.652867 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:31:42.652930 6494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.648873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:35419 (local address 127.18.211.130:45313)
0811 22:31:42.649198 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.649239 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:31:42.649334 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:31:42.649582 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:31:42.649584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:31:42.649600 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:31:42.649689 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.649693 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.650562 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.650565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.651723 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:31:42.651738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.652556 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.652559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:31:42.652685 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.652706 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:31:42.652770 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 22:31:42.652822 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":1,"thread_start_us":130,"threads_started":1}
I20260811 22:31:42.653716 6356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32a55f5de4ff0a8c151f4a0a5b537" candidate_uuid: "0c4263e82c1744fc9017e24599ed7eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22676fbd58a34ee08a80b98f9f169f51" is_pre_election: true
I20260811 22:31:42.653872 5845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32a55f5de4ff0a8c151f4a0a5b537" candidate_uuid: "0c4263e82c1744fc9017e24599ed7eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" is_pre_election: true
I20260811 22:31:42.654066 6356 raft_consensus.cc:2454] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4263e82c1744fc9017e24599ed7eb5 in term 0.
I20260811 22:31:42.654199 5845 raft_consensus.cc:2454] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4263e82c1744fc9017e24599ed7eb5 in term 0.
I20260811 22:31:42.654567 5976 leader_election.cc:302] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4263e82c1744fc9017e24599ed7eb5, 22676fbd58a34ee08a80b98f9f169f51; no voters:
I20260811 22:31:42.654906 6489 raft_consensus.cc:2781] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:31:42.655014 6489 raft_consensus.cc:482] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:31:42.655073 6489 raft_consensus.cc:3037] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:42.656289 6489 raft_consensus.cc:504] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.656666 6489 leader_election.cc:288] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [CANDIDATE]: Term 1 election: Requested vote from peers 22676fbd58a34ee08a80b98f9f169f51 (127.18.211.131:33239), 4a142128d78e4e7bba509f1fcc4cd10d (127.18.211.129:35419)
I20260811 22:31:42.657032 6356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32a55f5de4ff0a8c151f4a0a5b537" candidate_uuid: "0c4263e82c1744fc9017e24599ed7eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22676fbd58a34ee08a80b98f9f169f51"
I20260811 22:31:42.657176 5845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32a55f5de4ff0a8c151f4a0a5b537" candidate_uuid: "0c4263e82c1744fc9017e24599ed7eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a142128d78e4e7bba509f1fcc4cd10d"
I20260811 22:31:42.657187 6356 raft_consensus.cc:3037] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:42.657352 5845 raft_consensus.cc:3037] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:31:42.658371 6356 raft_consensus.cc:2454] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4263e82c1744fc9017e24599ed7eb5 in term 1.
I20260811 22:31:42.658681 5845 raft_consensus.cc:2454] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4263e82c1744fc9017e24599ed7eb5 in term 1.
I20260811 22:31:42.658681 5976 leader_election.cc:302] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4263e82c1744fc9017e24599ed7eb5, 22676fbd58a34ee08a80b98f9f169f51; no voters:
I20260811 22:31:42.658867 6489 raft_consensus.cc:2781] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:31:42.659116 6489 raft_consensus.cc:686] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 1 LEADER]: Becoming Leader. State: Replica: 0c4263e82c1744fc9017e24599ed7eb5, State: Running, Role: LEADER
I20260811 22:31:42.659493 6489 consensus_queue.cc:237] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } }
I20260811 22:31:42.662994 5582 catalog_manager.cc:5495] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c4263e82c1744fc9017e24599ed7eb5 (127.18.211.130). New cstate: current_term: 1 leader_uuid: "0c4263e82c1744fc9017e24599ed7eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c4263e82c1744fc9017e24599ed7eb5" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41915 } health_report { overall_health: HEALTHY } } }
I20260811 22:31:42.678996 6472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.675314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.18.211.130:41915)
0811 22:31:42.675451 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:31:42.675455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:31:42.675469 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:31:42.675660 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:31:42.675663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:31:42.675725 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:31:42.675833 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:31:42.676099 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.676606 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.677516 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:31:42.677754 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:31:42.677865 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:31:42.678173 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:31:42.678571 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:31:42.678708 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:31:42.678772 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 22:31:42.678826 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:31:42.684664 6356 raft_consensus.cc:1261] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 1 FOLLOWER]: Refusing update from remote peer 0c4263e82c1744fc9017e24599ed7eb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:42.684706 5845 raft_consensus.cc:1261] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 4a142128d78e4e7bba509f1fcc4cd10d [term 1 FOLLOWER]: Refusing update from remote peer 0c4263e82c1744fc9017e24599ed7eb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:31:42.685469 6489 consensus_queue.cc:1035] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a142128d78e4e7bba509f1fcc4cd10d" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:42.685662 6495 consensus_queue.cc:1035] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22676fbd58a34ee08a80b98f9f169f51" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 33239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:31:42.694048 6502 mvcc.cc:205] Tried to move back new op lower bound from 7317452810989252608 to 7317452810896310272. Current Snapshot: MvccSnapshot[applied={T|T < 7317452810989252608}]
I20260811 22:31:42.702574 5582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:42.702762 5582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:31:42.705283 5582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8da7b642c544d019f745a72a3130ea8: Sending DeleteTablet for 3 replicas of tablet 24e32a55f5de4ff0a8c151f4a0a5b537
I20260811 22:31:42.706036 6318 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e32a55f5de4ff0a8c151f4a0a5b537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:42 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:42186
I20260811 22:31:42.706127 5814 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e32a55f5de4ff0a8c151f4a0a5b537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:42 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:59008
I20260811 22:31:42.706189 6068 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e32a55f5de4ff0a8c151f4a0a5b537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:31:42 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:42036
I20260811 22:31:42.706755 6521 tablet_replica.cc:330] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: stopping tablet replica
I20260811 22:31:42.706872 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 5692
I20260811 22:31:42.712368 6521 raft_consensus.cc:2229] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:31:42.712818 6521 raft_consensus.cc:2258] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:42.719755 6521 ts_tablet_manager.cc:1905] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:42.722980 6520 tablet_replica.cc:330] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: stopping tablet replica
I20260811 22:31:42.723304 6521 ts_tablet_manager.cc:1918] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:42.723491 6521 log.cc:1196] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/24e32a55f5de4ff0a8c151f4a0a5b537
I20260811 22:31:42.723925 6520 raft_consensus.cc:2229] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:31:42.724409 6521 ts_tablet_manager.cc:1939] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 0c4263e82c1744fc9017e24599ed7eb5: Deleting consensus metadata
I20260811 22:31:42.724871 6520 raft_consensus.cc:2258] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:31:42.728731 6520 ts_tablet_manager.cc:1905] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:31:42.733338 5559 catalog_manager.cc:4841] TS 0c4263e82c1744fc9017e24599ed7eb5 (127.18.211.130:41915): tablet 24e32a55f5de4ff0a8c151f4a0a5b537 (table test-table [id=3e8bbe2de0f34b2ea38ba30aef253741]) successfully deleted
I20260811 22:31:42.734695 6520 ts_tablet_manager.cc:1918] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:31:42.734807 6520 log.cc:1196] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/24e32a55f5de4ff0a8c151f4a0a5b537
I20260811 22:31:42.735148 6520 ts_tablet_manager.cc:1939] T 24e32a55f5de4ff0a8c151f4a0a5b537 P 22676fbd58a34ee08a80b98f9f169f51: Deleting consensus metadata
I20260811 22:31:42.736541 5559 catalog_manager.cc:4841] TS 22676fbd58a34ee08a80b98f9f169f51 (127.18.211.131:33239): tablet 24e32a55f5de4ff0a8c151f4a0a5b537 (table test-table [id=3e8bbe2de0f34b2ea38ba30aef253741]) successfully deleted
W20260811 22:31:42.740963 5559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:35419 (error 108)
I20260811 22:31:42.741062 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 5948
I20260811 22:31:42.744228 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:31:42.742155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:35419 (local address 127.0.0.1:59016)
0811 22:31:42.744081 (+ 1926us) negotiation.cc:107] Waiting for socket to connect
0811 22:31:42.744148 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1837}
W20260811 22:31:42.747064 5559 catalog_manager.cc:4584] TS 4a142128d78e4e7bba509f1fcc4cd10d (127.18.211.129:35419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24e32a55f5de4ff0a8c151f4a0a5b537: Network error: Client connection negotiation failed: client connection to 127.18.211.129:35419: connect: Connection refused (error 111)
I20260811 22:31:42.753114 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 6200
I20260811 22:31:42.765733 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 5526
2026-08-11T22:31:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2604 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[6530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[6530](info): set up 2 sockets
Aug 11 22:31:42 dist-test-slave-8hhm krb5kdc[6530](info): commencing operation
krb5kdc: starting...
W20260811 22:31:44.093374 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:31:44 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:31:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:31:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/rangeradmin_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/rangeradmin_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/rangerlookup_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/rangerlookup_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
I20260811 22:31:44.279945 19278 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskG_0iNn/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/postgres -l logfile start
2026-08-11 22:31:46.857 UTC [6594] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:31:46.857 UTC [6594] LOG: listening on IPv4 address "127.18.211.148", port 58669
2026-08-11 22:31:46.860 UTC [6594] LOG: listening on Unix socket "/tmp/.s.PGSQL.58669"
2026-08-11 22:31:46.888 UTC [6598] LOG: database system was shut down at 2026-08-11 22:31:45 UTC
2026-08-11 22:31:46.896 UTC [6594] LOG: database system is ready to accept connections
I20260811 22:31:48.102262 19278 mini_postgres.cc:96] Postgres bound to 58669
2026-08-11 22:31:48.110 UTC [6615] FATAL: database "slave" does not exist
127.18.211.148:58669 - accepting connections
I20260811 22:31:48.111582 19278 mini_ranger.cc:161] Starting Ranger...
I20260811 22:31:48.128459 19278 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:31:48.625607 19278 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:31:48.625711 19278 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-admin
2026-08-11 22:31:48,948 [I] DB FLAVOR :POSTGRES
2026-08-11 22:31:48,949 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:31:48,949 [I] Checking connection..
2026-08-11 22:31:48,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:31:49,189 [I] Checking connection passed.
2026-08-11 22:31:49,189 [I] --------- Verifying version history table ---------
2026-08-11 22:31:49,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:31:49,448 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:31:49,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:31:49,703 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:31:49,704 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:31:49,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:31:49.959 UTC [6730] WARNING: there is no transaction in progress
2026-08-11 22:31:49,968 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:31:49,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:31:50,207 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:31:50,207 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:31:50,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:31:50,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:31:50,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8hhm.c.gcp-upstream.internal','N') ;"
2026-08-11 22:31:50,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:31:51,267 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:31:51,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:31:51,546 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:31:51,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:31:51,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:31:52,077 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:31:52,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:31:53.064 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.081 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.097 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.114 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.131 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.151 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.213 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.221 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.236 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.247 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.257 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.272 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.282 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.288 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.296 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53.304 UTC [6925] WARNING: there is no transaction in progress
2026-08-11 22:31:53,961 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:31:53,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:31:54,221 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:31:54,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:31:54,474 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:31:54,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:31:54,733 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:31:54,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:31:54,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:31:55,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8hhm.c.gcp-upstream.internal';"
2026-08-11 22:31:55,517 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:31:55,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:31:55,789 [I] DB_PATCHES have already been applied
I20260811 22:31:55.795781 19278 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-admin:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskG_0iNn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:31:55.795912 19278 mini_ranger.cc:193] Using host: 127.18.211.148
I20260811 22:31:55.800705 19278 mini_ranger.cc:227] Ranger admin URL: http://127.18.211.148:33295
Aug 11, 2026 10:31:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:31:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:31:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:31:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:31:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:31:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:31:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:31:57.235464 19278 mini_ranger.cc:160] Time spent starting Ranger: real 9.124s user 0.000s sys 0.013s
Aug 11, 2026 10:32:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:32:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:32:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:32:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:32:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:32:19.610607 19278 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/rangerkms_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/rangerkms_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.211.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.211.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:32:19 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487539, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:32:19.724992 19278 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:32:19.738008 19278 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:32:20.605037 19278 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:32:20.605149 19278 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms
I20260811 22:32:20.605288 19278 mini_ranger_kms.cc:227] Using postgres at 127.18.211.148:58669
2026-08-11 22:32:20,943 [I] DB FLAVOR :POSTGRES
2026-08-11 22:32:20,943 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:32:20,943 [I] Checking connection
2026-08-11 22:32:20,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:32:21,206 [I] connection success
2026-08-11 22:32:21,206 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:32:21,206 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:32:21,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:32:21,441 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:32:21,442 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:32:21,442 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:32:21,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:58669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:32:21,724 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:32:22.188632 19278 mini_ranger_kms.cc:310] Created kms service
I20260811 22:32:22.367364 19278 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:32:22.414397 19278 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:32:22 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487539, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:32:22.865298 19278 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:32:22.865476 19278 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms:/tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:32:22.865505 19278 mini_ranger_kms.cc:240] Using host: 127.18.211.148
I20260811 22:32:22.870733 19278 mini_ranger_kms.cc:276] Ranger KMS PID: 7240
I20260811 22:32:22.870887 19278 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.211.148:60675
Aug 11, 2026 10:32:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:32:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:32:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:32:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:32:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:32:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:32:25 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487545, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.211.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:32:25 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487545, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.211.148@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:32:25.820087 19278 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.095s user 0.003s sys 0.267s
I20260811 22:32:25.820242 19278 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:32:25.820355 19278 mini_ranger_kms.cc:401] 127.18.211.148:60675/kms/v1/keys
Aug 11 22:32:26 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487545, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.211.148@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
Aug 11 22:32:28 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:32:28.390799 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:46589
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:46589
--ranger_config_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:60675/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:32:28.510231 7333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:32:28.510540 7333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:32:28.510615 7333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:32:28.514654 7333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:32:28.514743 7333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:32:28.514782 7333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:32:28.514838 7333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:32:28.514875 7333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:32:28.520479 7333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.211.148:60675/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:46589
--ranger_config_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:46589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:32:28.521646 7333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:32:28.522675 7333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:32:28.533473 7348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:32:28.533674 7345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:32:28.533645 7333 server_base.cc:1034] running on GCE node
W20260811 22:32:28.533540 7343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:32:28.534402 7333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:32:28.535742 7333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:32:28.536950 7333 hybrid_clock.cc:648] HybridClock initialized: now 1786487548536931 us; error 52 us; skew 500 ppm
Aug 11 22:32:28 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487548, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:32:28.540570 7333 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:32:28.541569 7333 webserver.cc:460] Webserver started at http://127.18.211.190:44311/ using document root <none> and password file <none>
I20260811 22:32:28.542130 7333 fs_manager.cc:359] Metadata directory not provided
I20260811 22:32:28.542212 7333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:32:28.542450 7333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:32:28 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487548, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:32:28.604198 7333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "3fb5cefee9204f42b001a91014f24a9a"
format_stamp: "Formatted at 2026-08-11 22:32:28 on dist-test-slave-8hhm"
server_key: "6fe9319b235a494615297739af12aaab"
server_key_iv: "58b291b1c9ca51e63adba01b52f3956c"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:28.604918 7333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "3fb5cefee9204f42b001a91014f24a9a"
format_stamp: "Formatted at 2026-08-11 22:32:28 on dist-test-slave-8hhm"
server_key: "6fe9319b235a494615297739af12aaab"
server_key_iv: "58b291b1c9ca51e63adba01b52f3956c"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:28.609467 7333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:32:28.626966 7360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:28.628494 7333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:32:28.628654 7333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "3fb5cefee9204f42b001a91014f24a9a"
format_stamp: "Formatted at 2026-08-11 22:32:28 on dist-test-slave-8hhm"
server_key: "6fe9319b235a494615297739af12aaab"
server_key_iv: "58b291b1c9ca51e63adba01b52f3956c"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:28.628818 7333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:32:28.642114 7333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:32:28.645200 7333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:32:28.645468 7333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:32:28.654342 7333 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:46589
I20260811 22:32:28.654395 7467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:46589 every 8 connection(s)
I20260811 22:32:28.655364 7333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:32:28.658977 7469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:32:28.659381 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 7333
I20260811 22:32:28.659536 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:32:28.659859 19278 external_mini_cluster.cc:1428] Setting key 45c31bb10970636c3f035d1385388081
I20260811 22:32:28.666780 7469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a: Bootstrap starting.
Aug 11 22:32:28 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:32:28.669570 7469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a: Neither blocks nor log segments found. Creating new log.
I20260811 22:32:28.670971 7469 log.cc:824] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:32:28.673296 7469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a: No bootstrap required, opened a new log
I20260811 22:32:28.674427 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:28.661392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.18.211.190:46589)
0811 22:32:28.661825 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:28.661848 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:28.661884 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:32:28.662587 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:28.662604 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:28.662925 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:28.663273 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:32:28.664112 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:28.665136 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:28.666032 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:28.666524 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:28.666698 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:28.669330 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:32:28.669335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:32:28.669349 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:32:28.669376 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:32:28.671112 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:28.671654 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:28.671660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:28.671666 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:28.671830 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:28.672163 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:28.672166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:28.672168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:28.672660 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:32:28.672859 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:28.673195 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:32:28.673496 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":143,"threads_started":1}
I20260811 22:32:28.676789 7469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb5cefee9204f42b001a91014f24a9a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46589 } }
I20260811 22:32:28.677145 7469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:32:28.677263 7469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb5cefee9204f42b001a91014f24a9a, State: Initialized, Role: FOLLOWER
I20260811 22:32:28.677780 7469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb5cefee9204f42b001a91014f24a9a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46589 } }
I20260811 22:32:28.677953 7469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:32:28.678048 7469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:32:28.678159 7469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:32:28.679360 7469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb5cefee9204f42b001a91014f24a9a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46589 } }
I20260811 22:32:28.679749 7469 leader_election.cc:302] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fb5cefee9204f42b001a91014f24a9a; no voters:
I20260811 22:32:28.680131 7469 leader_election.cc:288] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:32:28.680719 7476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:32:28.680956 7476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [term 1 LEADER]: Becoming Leader. State: Replica: 3fb5cefee9204f42b001a91014f24a9a, State: Running, Role: LEADER
I20260811 22:32:28.681282 7476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb5cefee9204f42b001a91014f24a9a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46589 } }
I20260811 22:32:28.681625 7469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:32:28.682513 7476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fb5cefee9204f42b001a91014f24a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb5cefee9204f42b001a91014f24a9a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46589 } } }
I20260811 22:32:28.683137 7476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [sys.catalog]: This master's current role is: LEADER
I20260811 22:32:28.683344 7476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fb5cefee9204f42b001a91014f24a9a. Latest consensus state: current_term: 1 leader_uuid: "3fb5cefee9204f42b001a91014f24a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb5cefee9204f42b001a91014f24a9a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46589 } } }
I20260811 22:32:28.683602 7476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a [sys.catalog]: This master's current role is: LEADER
I20260811 22:32:28.684122 7469 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:32:28.689560 7494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:32:28.696043 7494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:32:28.703483 7494 catalog_manager.cc:1347] Generated new cluster ID: 3479a92a67a140a79302b2c325127807
I20260811 22:32:28.703665 7494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:32:28.716799 7494 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:32:28.718304 7494 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:32:28.726285 7494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a: Generated new TSK 0
I20260811 22:32:28.727228 7494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:32:29.552955 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:60675/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46589
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:32:29.708362 7546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:32:29.708642 7546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:32:29.708725 7546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:32:29.713814 7546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:32:29.713886 7546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:32:29.713969 7546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:32:29.719349 7546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.211.148:60675/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:32:29.720669 7546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:32:29.722119 7546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:32:29.734097 7563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:32:29.734282 7559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:32:29.734153 7561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:32:29.735659 7546 server_base.cc:1034] running on GCE node
I20260811 22:32:29.736116 7546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:32:29.736852 7546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:32:29.738054 7546 hybrid_clock.cc:648] HybridClock initialized: now 1786487549738028 us; error 41 us; skew 500 ppm
Aug 11 22:32:29 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:32:29.741869 7546 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:32:29.743038 7546 webserver.cc:460] Webserver started at http://127.18.211.129:34657/ using document root <none> and password file <none>
I20260811 22:32:29.743747 7546 fs_manager.cc:359] Metadata directory not provided
I20260811 22:32:29.743849 7546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:32:29.744184 7546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:32:29 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:32:29.760771 7546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3"
format_stamp: "Formatted at 2026-08-11 22:32:29 on dist-test-slave-8hhm"
server_key: "6da1fb022096396ae4facdc175aed0c8"
server_key_iv: "2d2bd5ec0e9e5d6714852a7f38939b07"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:29.761657 7546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3"
format_stamp: "Formatted at 2026-08-11 22:32:29 on dist-test-slave-8hhm"
server_key: "6da1fb022096396ae4facdc175aed0c8"
server_key_iv: "2d2bd5ec0e9e5d6714852a7f38939b07"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:29.767694 7546 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 22:32:29.785071 7576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:29.786538 7546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 22:32:29.786684 7546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3"
format_stamp: "Formatted at 2026-08-11 22:32:29 on dist-test-slave-8hhm"
server_key: "6da1fb022096396ae4facdc175aed0c8"
server_key_iv: "2d2bd5ec0e9e5d6714852a7f38939b07"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:29.786829 7546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:32:29.807411 7546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:32:29.811390 7546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:32:29.811617 7546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:32:29.812443 7546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:32:29.813570 7546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:32:29.813637 7546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:29.813702 7546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:32:29.813730 7546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:29.825299 7546 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:34167
I20260811 22:32:29.826603 7546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:32:29.830211 7793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:34167 every 8 connection(s)
I20260811 22:32:29.832700 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 7546
I20260811 22:32:29.832827 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:32:29.833197 19278 external_mini_cluster.cc:1428] Setting key 478bd1280abc1340ced0e7eb5f84fae2
Aug 11 22:32:29 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:32:29.847993 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:29.831922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:54219 (local address 127.18.211.190:46589)
0811 22:32:29.832195 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:29.832200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:29.833982 (+ 1782us) server_negotiation.cc:407] Connection header received
0811 22:32:29.835361 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:29.835365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:29.835425 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:29.835539 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:32:29.837083 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:29.837986 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:29.839367 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:29.839611 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:29.839759 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:29.842716 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:32:29.842723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:32:29.842729 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:32:29.842763 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:32:29.844978 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:29.845795 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:29.845801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:29.845805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:29.845914 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:29.846410 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:29.846416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:29.846418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:29.846675 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:32:29.846778 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:29.847590 (+ 812us) server_negotiation.cc:299] Negotiation successful
0811 22:32:29.847744 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:32:29.850121 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:29.832433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46589 (local address 127.18.211.129:54219)
0811 22:32:29.833812 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:29.833876 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:29.835111 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:29.835746 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:29.835757 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:29.836140 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:32:29.836908 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:29.836931 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:29.838172 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:29.838176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:29.839223 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:29.839234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:29.839806 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:29.839810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:29.840014 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:29.840729 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:32:29.840745 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:32:29.842512 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:32:29.845153 (+ 2641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:32:29.845162 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:32:29.845202 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:32:29.845653 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:32:29.846044 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:32:29.846047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:32:29.846049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:32:29.846270 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:32:29.846858 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:32:29.846866 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:32:29.847263 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 22:32:29.848572 (+ 1309us) client_negotiation.cc:240] Negotiation successful
0811 22:32:29.848963 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1132,"thread_start_us":141,"threads_started":1}
I20260811 22:32:29.851784 7794 heartbeater.cc:347] Connected to a master server at 127.18.211.190:46589
I20260811 22:32:29.852056 7794 heartbeater.cc:464] Registering TS with master...
I20260811 22:32:29.852703 7794 heartbeater.cc:511] Master 127.18.211.190:46589 requested a full tablet report, sending...
I20260811 22:32:29.854574 7394 ts_manager.cc:194] Registered new tserver with Master: f9e7a800b1b44f23bcb4fee6cb4259f3 (127.18.211.129:34167)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:32:29.856071 7394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:54219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:32:29.880568 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:60675/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46589
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:32:29 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:32:30.011361 7805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:32:30.011647 7805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:32:30.011760 7805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:32:30.016064 7805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:32:30.016139 7805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:32:30.016223 7805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:32:30.023020 7805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.211.148:60675/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:32:30.024222 7805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:32:30.025566 7805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:32:30.036973 7814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:32:30.037328 7821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:32:30.037591 7816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:32:30.038326 7805 server_base.cc:1034] running on GCE node
I20260811 22:32:30.038793 7805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:32:30.039588 7805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:32:30.040791 7805 hybrid_clock.cc:648] HybridClock initialized: now 1786487550040767 us; error 40 us; skew 500 ppm
Aug 11 22:32:30 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:32:30.044592 7805 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:32:30.045758 7805 webserver.cc:460] Webserver started at http://127.18.211.130:36961/ using document root <none> and password file <none>
I20260811 22:32:30.046517 7805 fs_manager.cc:359] Metadata directory not provided
I20260811 22:32:30.046624 7805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:32:30.046908 7805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:32:30 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:32:30.063895 7805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "281006fd693f4c7bbc43badba08800a7"
format_stamp: "Formatted at 2026-08-11 22:32:30 on dist-test-slave-8hhm"
server_key: "f702a1991754a956c067ac357ed84cce"
server_key_iv: "51ec9363d14df447ca8e4b833755308f"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:30.064803 7805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "281006fd693f4c7bbc43badba08800a7"
format_stamp: "Formatted at 2026-08-11 22:32:30 on dist-test-slave-8hhm"
server_key: "f702a1991754a956c067ac357ed84cce"
server_key_iv: "51ec9363d14df447ca8e4b833755308f"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:30.070201 7805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 22:32:30.097164 7829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:30.098735 7805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:32:30.098865 7805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "281006fd693f4c7bbc43badba08800a7"
format_stamp: "Formatted at 2026-08-11 22:32:30 on dist-test-slave-8hhm"
server_key: "f702a1991754a956c067ac357ed84cce"
server_key_iv: "51ec9363d14df447ca8e4b833755308f"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:30.099013 7805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:32:30.120637 7805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:32:30.124282 7805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:32:30.124467 7805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:32:30.125156 7805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:32:30.126205 7805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:32:30.126269 7805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:30.126327 7805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:32:30.126358 7805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:30.139333 7805 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:42875
I20260811 22:32:30.140501 7805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:32:30.145608 8045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:42875 every 8 connection(s)
I20260811 22:32:30.150619 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 7805
I20260811 22:32:30.150763 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:32:30.151109 19278 external_mini_cluster.cc:1428] Setting key dd288bb33d7e837cea4d861f54f266e4
Aug 11 22:32:30 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:32:30.160698 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.144547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46589 (local address 127.18.211.130:50585)
0811 22:32:30.145192 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.145273 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:30.146580 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:30.147219 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:30.147228 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:30.147618 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:32:30.148293 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.148312 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.149545 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.149548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.150923 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.150934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.151736 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.151741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.151958 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.152998 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:32:30.153022 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:32:30.154774 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:32:30.157088 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:32:30.157094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:32:30.157111 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:32:30.157532 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:32:30.157896 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:32:30.157900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:32:30.157902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:32:30.158101 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:32:30.158656 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:32:30.158665 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:32:30.159006 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 22:32:30.159243 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 22:32:30.159610 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":227,"threads_started":1}
I20260811 22:32:30.162286 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.144007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:50585 (local address 127.18.211.190:46589)
0811 22:32:30.144140 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.144145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.145406 (+ 1261us) server_negotiation.cc:407] Connection header received
0811 22:32:30.146838 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.146841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.146903 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.147005 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:32:30.148474 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.149373 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.151064 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.151414 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.151509 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.154964 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:32:30.154968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:32:30.154971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:32:30.154997 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:32:30.156935 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:30.157666 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:30.157669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:30.157671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:30.157772 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:30.158239 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:30.158243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:30.158245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:30.158486 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:32:30.158592 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.161948 (+ 3356us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.162088 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:32:30.163249 8046 heartbeater.cc:347] Connected to a master server at 127.18.211.190:46589
I20260811 22:32:30.163554 8046 heartbeater.cc:464] Registering TS with master...
I20260811 22:32:30.164230 8046 heartbeater.cc:511] Master 127.18.211.190:46589 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:32:30.165455 7394 ts_manager.cc:194] Registered new tserver with Master: 281006fd693f4c7bbc43badba08800a7 (127.18.211.130:42875)
I20260811 22:32:30.166229 7394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:50585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:32:30.188160 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:60675/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46589
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:32:30.345981 8051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:32:30.346333 8051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:32:30.346393 8051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:32:30.352228 8051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:32:30.352305 8051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:32:30.352388 8051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:32:30.359864 8051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.211.148:60675/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:46589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:32:30.361163 8051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:32:30.362452 8051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:32:30.374734 8061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:32:30.375062 8057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:32:30.374719 8058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:32:30.377199 8051 server_base.cc:1034] running on GCE node
I20260811 22:32:30.377689 8051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:32:30.378438 8051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:32:30.379720 8051 hybrid_clock.cc:648] HybridClock initialized: now 1786487550379683 us; error 39 us; skew 500 ppm
Aug 11 22:32:30 dist-test-slave-8hhm krb5kdc[6530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:32:30.384146 8051 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:32:30.385274 8051 webserver.cc:460] Webserver started at http://127.18.211.131:40357/ using document root <none> and password file <none>
I20260811 22:32:30.385957 8051 fs_manager.cc:359] Metadata directory not provided
I20260811 22:32:30.386040 8051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:32:30.386264 8051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:32:30 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
Aug 11 22:32:30 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:32:30.412127 8051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "f4d16854917e4450b68cd1e25b8d665b"
format_stamp: "Formatted at 2026-08-11 22:32:30 on dist-test-slave-8hhm"
server_key: "3d8817d6a9534c2a547dd43b2cce9cd7"
server_key_iv: "e2c961dc901cf87432adb65ecb49ba3a"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:30.412864 8051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "f4d16854917e4450b68cd1e25b8d665b"
format_stamp: "Formatted at 2026-08-11 22:32:30 on dist-test-slave-8hhm"
server_key: "3d8817d6a9534c2a547dd43b2cce9cd7"
server_key_iv: "e2c961dc901cf87432adb65ecb49ba3a"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:30.426903 8051 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.007s sys 0.001s
I20260811 22:32:30.462383 8078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:30.466743 8051 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.003s sys 0.000s
I20260811 22:32:30.466915 8051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "f4d16854917e4450b68cd1e25b8d665b"
format_stamp: "Formatted at 2026-08-11 22:32:30 on dist-test-slave-8hhm"
server_key: "3d8817d6a9534c2a547dd43b2cce9cd7"
server_key_iv: "e2c961dc901cf87432adb65ecb49ba3a"
server_key_version: "kuduclusterkey@0"
I20260811 22:32:30.467090 8051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:32:30.487798 8051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:32:30.491943 8051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:32:30.492161 8051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:32:30.493026 8051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:32:30.494395 8051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:32:30.494463 8051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:30.494526 8051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:32:30.494560 8051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:32:30.517028 8051 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:42333
I20260811 22:32:30.519580 8051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:32:30.525401 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 8051
I20260811 22:32:30.525557 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:32:30.525941 19278 external_mini_cluster.cc:1428] Setting key 17a23dfc837966007e57fe1106e4b6fd
I20260811 22:32:30.527354 8278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:42333 every 8 connection(s)
Aug 11 22:32:30 dist-test-slave-8hhm krb5kdc[6530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:32:30.542696 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.525175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:51621 (local address 127.18.211.190:46589)
0811 22:32:30.525365 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.525371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.526041 (+ 670us) server_negotiation.cc:407] Connection header received
0811 22:32:30.528573 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.528577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.528658 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.528709 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:32:30.530595 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.531452 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.533869 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.534230 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.534336 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.537724 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:32:30.537728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:32:30.537731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:32:30.537759 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:32:30.540047 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:30.540759 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:30.540763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:30.540765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:30.540851 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:30.541296 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:30.541301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:30.541308 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:30.541576 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:32:30.541685 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.542379 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.542525 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:32:30.544600 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.523967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46589 (local address 127.18.211.131:51621)
0811 22:32:30.525849 (+ 1882us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.525909 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:30.528269 (+ 2360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:30.529183 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:30.529193 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:30.529642 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:32:30.530413 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.530435 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.531935 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.531938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.533723 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.533734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.534565 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.534569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.534775 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.535599 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:32:30.535620 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:32:30.537517 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:32:30.540247 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:32:30.540253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:32:30.540270 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:32:30.540634 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:32:30.540955 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:32:30.540958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:32:30.540960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:32:30.541139 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:32:30.541763 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:32:30.541769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:32:30.542081 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 22:32:30.543082 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0811 22:32:30.543420 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1626,"thread_start_us":197,"threads_started":1}
I20260811 22:32:30.546214 8279 heartbeater.cc:347] Connected to a master server at 127.18.211.190:46589
I20260811 22:32:30.546479 8279 heartbeater.cc:464] Registering TS with master...
I20260811 22:32:30.547078 8279 heartbeater.cc:511] Master 127.18.211.190:46589 requested a full tablet report, sending...
I20260811 22:32:30.548269 7394 ts_manager.cc:194] Registered new tserver with Master: f4d16854917e4450b68cd1e25b8d665b (127.18.211.131:42333)
I20260811 22:32:30.549058 7394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:51621
I20260811 22:32:30.554754 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:32:30.567732 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.557490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.18.211.190:46589)
0811 22:32:30.557601 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.557605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.558267 (+ 662us) server_negotiation.cc:407] Connection header received
0811 22:32:30.558444 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.558447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.558501 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.558548 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:32:30.559516 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.560231 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.561457 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.561776 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.561857 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.563815 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:32:30.563819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:32:30.563822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:32:30.563857 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:32:30.565666 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:30.566194 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:30.566198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:30.566199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:30.566292 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:32:30.566648 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:32:30.566652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:32:30.566654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:32:30.566906 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:32:30.567024 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.567438 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.567579 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:32:30.572379 7394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:32:30.576956 7394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:32:30.602360 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.593515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34167 (local address 127.0.0.1:48266)
0811 22:32:30.595937 (+ 2422us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.596003 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:30.596138 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:30.597442 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:30.597449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:30.597478 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:32:30.597617 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.597624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.599047 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.599051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.600438 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.600450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.601766 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.601769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.601893 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.601934 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:32:30.602065 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 22:32:30.602186 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2232,"thread_start_us":81,"threads_started":1}
I20260811 22:32:30.602985 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.594808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.18.211.129:34167)
0811 22:32:30.596790 (+ 1982us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.596799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.596823 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:32:30.596905 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.596910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.597149 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.597332 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:32:30.597767 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.598891 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.600584 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.601442 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.601564 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.601696 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.602748 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.602850 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1875,"thread_start_us":202,"threads_started":1}
I20260811 22:32:30.605516 8301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.593513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:42875 (local address 127.0.0.1:57540)
0811 22:32:30.596407 (+ 2894us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.596420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:30.596556 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:30.599157 (+ 2601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:30.599161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:30.599180 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:32:30.599293 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.599301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.600753 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.600759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.602893 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.602905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.604344 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.604347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.604452 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.604474 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:32:30.604544 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:32:30.604608 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2831,"mutex_wait_us":689,"thread_start_us":174,"threads_started":1}
I20260811 22:32:30.606046 8300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.593671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57540 (local address 127.18.211.130:42875)
0811 22:32:30.598470 (+ 4799us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.598485 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.598508 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:32:30.598583 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.598588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.598810 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.598990 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:32:30.599438 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.600613 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.603154 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.604001 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.604150 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.604279 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.605795 (+ 1516us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.605897 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4687,"thread_start_us":152,"threads_started":1}
I20260811 22:32:30.606207 7680 tablet_service.cc:1467] Processing CreateTablet for tablet 311503e00f2847cfb36a33ea3512eec8 (DEFAULT_TABLE table=test-table [id=56dcd8fd622946c1848da8c8c8d159e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:32:30.607402 7680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311503e00f2847cfb36a33ea3512eec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:32:30.608755 7902 tablet_service.cc:1467] Processing CreateTablet for tablet 311503e00f2847cfb36a33ea3512eec8 (DEFAULT_TABLE table=test-table [id=56dcd8fd622946c1848da8c8c8d159e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:32:30.610874 7902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311503e00f2847cfb36a33ea3512eec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:32:30.616127 8314 tablet_bootstrap.cc:492] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3: Bootstrap starting.
I20260811 22:32:30.618618 8314 tablet_bootstrap.cc:654] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:32:30.619441 8314 log.cc:824] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:32:30.621459 8314 tablet_bootstrap.cc:492] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3: No bootstrap required, opened a new log
I20260811 22:32:30.621654 8314 ts_tablet_manager.cc:1397] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:32:30.625554 8317 tablet_bootstrap.cc:492] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Bootstrap starting.
I20260811 22:32:30.626648 8314 raft_consensus.cc:348] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.626919 8314 raft_consensus.cc:374] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:32:30.626983 8314 raft_consensus.cc:729] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e7a800b1b44f23bcb4fee6cb4259f3, State: Initialized, Role: FOLLOWER
I20260811 22:32:30.627537 8314 consensus_queue.cc:260] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.628305 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.594311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:42333 (local address 127.0.0.1:47614)
0811 22:32:30.602538 (+ 8227us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.602553 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:30.602625 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:30.622582 (+ 19957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:30.622586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:30.622607 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:32:30.622721 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.622727 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.625231 (+ 2504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.625236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.626544 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.626558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.627832 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.627837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.627946 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.627982 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:32:30.628051 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:32:30.628136 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8173,"thread_start_us":102,"threads_started":1}
I20260811 22:32:30.628409 8317 tablet_bootstrap.cc:654] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Neither blocks nor log segments found. Creating new log.
I20260811 22:32:30.628747 8314 ts_tablet_manager.cc:1428] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:32:30.629113 7794 heartbeater.cc:503] Master 127.18.211.190:46589 was elected leader, sending a full tablet report...
I20260811 22:32:30.629163 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.594762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.18.211.131:42333)
0811 22:32:30.622071 (+ 27309us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.622084 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.622113 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:32:30.622199 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.622205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.622406 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.623633 (+ 1227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:32:30.623782 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.625064 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.626775 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.627671 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.628690 (+ 1019us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.628837 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.628895 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.628998 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27169,"thread_start_us":173,"threads_started":1}
I20260811 22:32:30.629412 8317 log.cc:824] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:32:30.632351 8158 tablet_service.cc:1467] Processing CreateTablet for tablet 311503e00f2847cfb36a33ea3512eec8 (DEFAULT_TABLE table=test-table [id=56dcd8fd622946c1848da8c8c8d159e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:32:30.633615 8158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311503e00f2847cfb36a33ea3512eec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:32:30.633879 8317 tablet_bootstrap.cc:492] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: No bootstrap required, opened a new log
I20260811 22:32:30.634053 8317 ts_tablet_manager.cc:1397] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 22:32:30.638096 8317 raft_consensus.cc:348] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.638429 8317 raft_consensus.cc:374] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:32:30.638547 8317 raft_consensus.cc:729] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 281006fd693f4c7bbc43badba08800a7, State: Initialized, Role: FOLLOWER
I20260811 22:32:30.639022 8317 consensus_queue.cc:260] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.640022 8317 ts_tablet_manager.cc:1428] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:32:30.640360 8046 heartbeater.cc:503] Master 127.18.211.190:46589 was elected leader, sending a full tablet report...
W20260811 22:32:30.642143 8047 tablet.cc:2369] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:32:30.643069 8323 tablet_bootstrap.cc:492] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Bootstrap starting.
I20260811 22:32:30.646275 8323 tablet_bootstrap.cc:654] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Neither blocks nor log segments found. Creating new log.
I20260811 22:32:30.647408 8323 log.cc:824] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:32:30.650115 8323 tablet_bootstrap.cc:492] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: No bootstrap required, opened a new log
I20260811 22:32:30.650349 8323 ts_tablet_manager.cc:1397] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:32:30.656169 8323 raft_consensus.cc:348] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.656505 8323 raft_consensus.cc:374] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:32:30.656580 8323 raft_consensus.cc:729] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d16854917e4450b68cd1e25b8d665b, State: Initialized, Role: FOLLOWER
I20260811 22:32:30.657140 8323 consensus_queue.cc:260] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.658335 8323 ts_tablet_manager.cc:1428] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 22:32:30.658790 8279 heartbeater.cc:503] Master 127.18.211.190:46589 was elected leader, sending a full tablet report...
I20260811 22:32:30.689918 8318 raft_consensus.cc:482] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:32:30.690275 8318 raft_consensus.cc:504] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.691809 8318 leader_election.cc:288] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 281006fd693f4c7bbc43badba08800a7 (127.18.211.130:42875), f4d16854917e4450b68cd1e25b8d665b (127.18.211.131:42333)
I20260811 22:32:30.698189 8300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.692178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:56809 (local address 127.18.211.130:42875)
0811 22:32:30.692301 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.692306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.693083 (+ 777us) server_negotiation.cc:407] Connection header received
0811 22:32:30.693194 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.693197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.693276 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.693368 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:32:30.694079 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.694867 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.696722 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.697341 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.697436 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.697484 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.697965 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.698046 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:32:30.698719 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.692061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:42875 (local address 127.18.211.129:56809)
0811 22:32:30.692970 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.692984 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:30.693096 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:30.693503 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:30.693506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:30.693526 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:32:30.693924 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.693933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.695049 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.695052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.696560 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.696572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.697662 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.697666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.697812 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.697836 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:32:30.698400 (+ 564us) client_negotiation.cc:240] Negotiation successful
0811 22:32:30.698475 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":134,"threads_started":1}
I20260811 22:32:30.698794 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.692697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:49573 (local address 127.18.211.131:42333)
0811 22:32:30.692823 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.692828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.693405 (+ 577us) server_negotiation.cc:407] Connection header received
0811 22:32:30.693583 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.693587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.693633 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.693678 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:32:30.694071 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.694810 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.696528 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.697082 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.697170 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.697240 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.698603 (+ 1363us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.698679 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:32:30.699362 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.692566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:42333 (local address 127.18.211.129:49573)
0811 22:32:30.693329 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.693341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:32:30.693463 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:32:30.693812 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:32:30.693815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:32:30.693830 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:32:30.693949 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.693954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.694967 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.694970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.696186 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:32:30.696197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.698160 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.698164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:32:30.698447 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.698472 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:32:30.699144 (+ 672us) client_negotiation.cc:240] Negotiation successful
0811 22:32:30.699216 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260811 22:32:30.699404 7944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311503e00f2847cfb36a33ea3512eec8" candidate_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "281006fd693f4c7bbc43badba08800a7" is_pre_election: true
I20260811 22:32:30.699783 7944 raft_consensus.cc:2454] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e7a800b1b44f23bcb4fee6cb4259f3 in term 0.
I20260811 22:32:30.699957 8190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311503e00f2847cfb36a33ea3512eec8" candidate_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d16854917e4450b68cd1e25b8d665b" is_pre_election: true
I20260811 22:32:30.700233 8190 raft_consensus.cc:2454] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e7a800b1b44f23bcb4fee6cb4259f3 in term 0.
I20260811 22:32:30.700337 7588 leader_election.cc:302] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 281006fd693f4c7bbc43badba08800a7, f9e7a800b1b44f23bcb4fee6cb4259f3; no voters:
I20260811 22:32:30.700639 8318 raft_consensus.cc:2781] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:32:30.700742 8318 raft_consensus.cc:482] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:32:30.700798 8318 raft_consensus.cc:3037] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:32:30.702220 8318 raft_consensus.cc:504] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.702587 8318 leader_election.cc:288] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [CANDIDATE]: Term 1 election: Requested vote from peers 281006fd693f4c7bbc43badba08800a7 (127.18.211.130:42875), f4d16854917e4450b68cd1e25b8d665b (127.18.211.131:42333)
I20260811 22:32:30.703277 7944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311503e00f2847cfb36a33ea3512eec8" candidate_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "281006fd693f4c7bbc43badba08800a7"
I20260811 22:32:30.703395 7944 raft_consensus.cc:3037] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:32:30.704682 7944 raft_consensus.cc:2454] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e7a800b1b44f23bcb4fee6cb4259f3 in term 1.
I20260811 22:32:30.705049 7588 leader_election.cc:302] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 281006fd693f4c7bbc43badba08800a7, f9e7a800b1b44f23bcb4fee6cb4259f3; no voters:
I20260811 22:32:30.705189 8190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311503e00f2847cfb36a33ea3512eec8" candidate_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d16854917e4450b68cd1e25b8d665b"
I20260811 22:32:30.705296 8318 raft_consensus.cc:2781] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:32:30.705328 8190 raft_consensus.cc:3037] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:32:30.705514 8318 raft_consensus.cc:686] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 1 LEADER]: Becoming Leader. State: Replica: f9e7a800b1b44f23bcb4fee6cb4259f3, State: Running, Role: LEADER
I20260811 22:32:30.705883 8318 consensus_queue.cc:237] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } }
I20260811 22:32:30.706600 8190 raft_consensus.cc:2454] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e7a800b1b44f23bcb4fee6cb4259f3 in term 1.
I20260811 22:32:30.709657 7394 catalog_manager.cc:5495] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e7a800b1b44f23bcb4fee6cb4259f3 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e7a800b1b44f23bcb4fee6cb4259f3" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 34167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 } health_report { overall_health: UNKNOWN } } }
I20260811 22:32:30.732304 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.727180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.18.211.129:34167)
0811 22:32:30.727680 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 22:32:30.727685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:32:30.727698 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:32:30.727774 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:32:30.727777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:32:30.727844 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:32:30.727955 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:32:30.728275 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.729065 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.730343 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:32:30.730674 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:32:30.730773 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:32:30.731209 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:32:30.731799 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:32:30.731920 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:32:30.732118 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 22:32:30.732179 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384}
I20260811 22:32:30.740399 8190 raft_consensus.cc:1261] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 1 FOLLOWER]: Refusing update from remote peer f9e7a800b1b44f23bcb4fee6cb4259f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:32:30.741187 8318 consensus_queue.cc:1035] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d16854917e4450b68cd1e25b8d665b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:32:30.741322 7944 raft_consensus.cc:1261] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 1 FOLLOWER]: Refusing update from remote peer f9e7a800b1b44f23bcb4fee6cb4259f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:32:30.741873 8318 consensus_queue.cc:1035] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "281006fd693f4c7bbc43badba08800a7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:32:30.751041 8339 mvcc.cc:205] Tried to move back new op lower bound from 7317453007824396288 to 7317453007694544896. Current Snapshot: MvccSnapshot[applied={T|T < 7317453007824396288}]
I20260811 22:32:30.754387 8342 mvcc.cc:205] Tried to move back new op lower bound from 7317453007824396288 to 7317453007694544896. Current Snapshot: MvccSnapshot[applied={T|T < 7317453007824396288}]
I20260811 22:32:30.763422 7392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:32:30.763610 7392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:32:30.766253 7392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fb5cefee9204f42b001a91014f24a9a: Sending DeleteTablet for 3 replicas of tablet 311503e00f2847cfb36a33ea3512eec8
I20260811 22:32:30.767490 7680 tablet_service.cc:1514] Processing DeleteTablet for tablet 311503e00f2847cfb36a33ea3512eec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:32:30 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:48266
I20260811 22:32:30.767643 8158 tablet_service.cc:1514] Processing DeleteTablet for tablet 311503e00f2847cfb36a33ea3512eec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:32:30 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:47614
I20260811 22:32:30.768213 8351 tablet_replica.cc:330] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3: stopping tablet replica
I20260811 22:32:30.768527 8352 tablet_replica.cc:330] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: stopping tablet replica
I20260811 22:32:30.768607 8351 raft_consensus.cc:2229] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:32:30.768723 7902 tablet_service.cc:1514] Processing DeleteTablet for tablet 311503e00f2847cfb36a33ea3512eec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:32:30 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:57540
I20260811 22:32:30.768994 8352 raft_consensus.cc:2229] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:32:30.769022 8351 raft_consensus.cc:2258] T 311503e00f2847cfb36a33ea3512eec8 P f9e7a800b1b44f23bcb4fee6cb4259f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:32:30.769398 8352 raft_consensus.cc:2258] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:32:30.769976 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 7546
I20260811 22:32:30.777462 8353 tablet_replica.cc:330] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: stopping tablet replica
I20260811 22:32:30.777805 8353 raft_consensus.cc:2229] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:32:30.778065 8353 raft_consensus.cc:2258] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:32:30.779752 8352 ts_tablet_manager.cc:1905] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:32:30.784461 8352 ts_tablet_manager.cc:1918] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:32:30.784584 8352 log.cc:1196] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/311503e00f2847cfb36a33ea3512eec8
I20260811 22:32:30.785009 8352 ts_tablet_manager.cc:1939] T 311503e00f2847cfb36a33ea3512eec8 P f4d16854917e4450b68cd1e25b8d665b: Deleting consensus metadata
I20260811 22:32:30.786402 7361 catalog_manager.cc:4841] TS f4d16854917e4450b68cd1e25b8d665b (127.18.211.131:42333): tablet 311503e00f2847cfb36a33ea3512eec8 (table test-table [id=56dcd8fd622946c1848da8c8c8d159e4]) successfully deleted
I20260811 22:32:30.791301 8353 ts_tablet_manager.cc:1905] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:32:30.796214 8353 ts_tablet_manager.cc:1918] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:32:30.796368 8353 log.cc:1196] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/311503e00f2847cfb36a33ea3512eec8
I20260811 22:32:30.796793 8353 ts_tablet_manager.cc:1939] T 311503e00f2847cfb36a33ea3512eec8 P 281006fd693f4c7bbc43badba08800a7: Deleting consensus metadata
W20260811 22:32:30.796757 7361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:34167 (error 108)
I20260811 22:32:30.797072 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 7805
I20260811 22:32:30.803242 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.802782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:34167 (local address 127.0.0.1:48288)
0811 22:32:30.802976 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.803101 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:34167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 22:32:30.803556 7361 catalog_manager.cc:4584] TS f9e7a800b1b44f23bcb4fee6cb4259f3 (127.18.211.129:34167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 311503e00f2847cfb36a33ea3512eec8: Network error: Client connection negotiation failed: client connection to 127.18.211.129:34167: connect: Connection refused (error 111)
I20260811 22:32:30.810014 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 8051
I20260811 22:32:30.819687 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:32:30.819260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:42875 (local address 127.0.0.1:57542)
0811 22:32:30.819496 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:32:30.819569 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.130:42875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260811 22:32:30.830327 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 7333
I20260811 22:32:30.843190 19278 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:32:31.189497 19278 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:32:31.189625 19278 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:32:31.534380 19278 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:32:31.534 UTC [6594] LOG: received smart shutdown request
2026-08-11 22:32:31.538 UTC [6594] LOG: background worker "logical replication launcher" (PID 6610) exited with exit code 1
2026-08-11 22:32:31.539 UTC [6599] LOG: shutting down
2026-08-11 22:32:31.707 UTC [6594] LOG: database system is shut down
2026-08-11T22:32:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49059 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:32:31 dist-test-slave-8hhm krb5kdc[8367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:32:31 dist-test-slave-8hhm krb5kdc[8367](info): set up 2 sockets
Aug 11 22:32:31 dist-test-slave-8hhm krb5kdc[8367](info): commencing operation
krb5kdc: starting...
W20260811 22:32:33.127588 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:32:33 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:32:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:32:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/rangeradmin_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/rangeradmin_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/rangerlookup_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/rangerlookup_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
I20260811 22:32:33.305061 19278 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskG_0iNn/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/postgres -l logfile start
2026-08-11 22:32:35.848 UTC [8439] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:32:35.849 UTC [8439] LOG: listening on IPv4 address "127.18.211.148", port 49475
2026-08-11 22:32:35.852 UTC [8439] LOG: listening on Unix socket "/tmp/.s.PGSQL.49475"
2026-08-11 22:32:35.874 UTC [8447] LOG: database system was shut down at 2026-08-11 22:32:34 UTC
2026-08-11 22:32:35.878 UTC [8439] LOG: database system is ready to accept connections
I20260811 22:32:37.099216 19278 mini_postgres.cc:96] Postgres bound to 49475
2026-08-11 22:32:37.106 UTC [8465] FATAL: database "slave" does not exist
127.18.211.148:49475 - accepting connections
I20260811 22:32:37.107441 19278 mini_ranger.cc:161] Starting Ranger...
I20260811 22:32:37.124439 19278 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:32:37.580166 19278 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:32:37.580248 19278 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-admin
2026-08-11 22:32:37,871 [I] DB FLAVOR :POSTGRES
2026-08-11 22:32:37,871 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:32:37,871 [I] Checking connection..
2026-08-11 22:32:37,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:32:38,139 [I] Checking connection passed.
2026-08-11 22:32:38,140 [I] --------- Verifying version history table ---------
2026-08-11 22:32:38,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:32:38,400 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:32:38,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:32:38,671 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:32:38,671 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:32:38,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:32:38.935 UTC [8585] WARNING: there is no transaction in progress
2026-08-11 22:32:38,945 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:32:38,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:32:39,217 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:32:39,217 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:32:39,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:32:39,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:32:39,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8hhm.c.gcp-upstream.internal','N') ;"
2026-08-11 22:32:39,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:32:40,243 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:32:40,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:32:40,514 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:32:40,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:32:40,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:32:41,042 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:32:41,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:32:41.998 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.014 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.030 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.045 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.061 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.081 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.135 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.141 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.156 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.166 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.176 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.190 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.200 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.206 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.213 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42.220 UTC [8774] WARNING: there is no transaction in progress
2026-08-11 22:32:42,853 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:32:42,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:32:43,109 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:32:43,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:32:43,377 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:32:43,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:32:43,629 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:32:43,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:32:43,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:32:44,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8hhm.c.gcp-upstream.internal';"
2026-08-11 22:32:44,393 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:32:44,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:32:44,682 [I] DB_PATCHES have already been applied
I20260811 22:32:44.688097 19278 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-admin:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskG_0iNn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:32:44.688165 19278 mini_ranger.cc:193] Using host: 127.18.211.148
I20260811 22:32:44.692960 19278 mini_ranger.cc:227] Ranger admin URL: http://127.18.211.148:46973
Aug 11, 2026 10:32:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:32:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:32:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:32:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:32:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:32:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:32:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:32:46.179560 19278 mini_ranger.cc:160] Time spent starting Ranger: real 9.072s user 0.002s sys 0.009s
Aug 11, 2026 10:33:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:33:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 10:33:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:33:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:33:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:33:07.915984 19278 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.211.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.211.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/rangerkms_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.211.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/rangerkms_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.211.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.211.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.211.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/HTTP_127.18.211.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:33:08 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487588, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:33:08.011173 19278 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:33:08.022425 19278 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:33:08.862460 19278 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:33:08.862536 19278 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms
I20260811 22:33:08.862550 19278 mini_ranger_kms.cc:227] Using postgres at 127.18.211.148:49475
2026-08-11 22:33:09,052 [I] DB FLAVOR :POSTGRES
2026-08-11 22:33:09,052 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:33:09,052 [I] Checking connection
2026-08-11 22:33:09,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:33:09,292 [I] connection success
2026-08-11 22:33:09,292 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:33:09,292 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:33:09,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:33:09,538 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:33:09,538 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:33:09,538 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:33:09,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.211.148:49475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:33:09,794 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:33:10.228752 19278 mini_ranger_kms.cc:310] Created kms service
I20260811 22:33:10.435791 19278 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:33:10.479794 19278 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:33:10 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487588, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:10.871387 19278 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:33:10.871543 19278 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms:/tmp/dist-test-taskG_0iNn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskG_0iNn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:33:10.871572 19278 mini_ranger_kms.cc:240] Using host: 127.18.211.148
I20260811 22:33:10.876475 19278 mini_ranger_kms.cc:276] Ranger KMS PID: 9078
I20260811 22:33:10.876565 19278 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.211.148:42693
Aug 11, 2026 10:33:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:33:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:33:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:33:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:33:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:33:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:33:13 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.211.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:13.810446 19278 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.799s user 0.007s sys 0.167s
I20260811 22:33:13.810592 19278 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:33:13.810665 19278 mini_ranger_kms.cc:401] 127.18.211.148:42693/kms/v1/keys
Aug 11 22:33:13 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.211.148@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
Aug 11 22:33:14 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.211.148@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
Aug 11 22:33:16 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:33:16.590528 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43607
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:43607
--ranger_config_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:16.718997 9153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:16.719269 9153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:16.719308 9153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:16.723273 9153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:16.723328 9153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:16.723347 9153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:16.723363 9153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:16.723378 9153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:16.728711 9153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.211.148:42693/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:43607
--ranger_config_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:16.729746 9153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:16.730693 9153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:16.739513 9166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:16.739497 9161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:16.739609 9153 server_base.cc:1034] running on GCE node
W20260811 22:33:16.739497 9162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:16.740330 9153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:16.741528 9153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:16.742690 9153 hybrid_clock.cc:648] HybridClock initialized: now 1786487596742662 us; error 39 us; skew 500 ppm
Aug 11 22:33:16 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487596, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:16.745538 9153 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:16.746723 9153 webserver.cc:460] Webserver started at http://127.18.211.190:42005/ using document root <none> and password file <none>
I20260811 22:33:16.747299 9153 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:16.747357 9153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:16.747591 9153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:33:16 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487596, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:16.795719 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "f1a08fb70ecb4aaf993be032843b759f"
format_stamp: "Formatted at 2026-08-11 22:33:16 on dist-test-slave-8hhm"
server_key: "22feb46d048fdf2675eee7419b532d03"
server_key_iv: "7e10d35299c8768261e4a3e7e03aff4f"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:16.796456 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "f1a08fb70ecb4aaf993be032843b759f"
format_stamp: "Formatted at 2026-08-11 22:33:16 on dist-test-slave-8hhm"
server_key: "22feb46d048fdf2675eee7419b532d03"
server_key_iv: "7e10d35299c8768261e4a3e7e03aff4f"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:16.800720 9153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:33:16.814675 9180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:16.816120 9153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:33:16.816277 9153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "f1a08fb70ecb4aaf993be032843b759f"
format_stamp: "Formatted at 2026-08-11 22:33:16 on dist-test-slave-8hhm"
server_key: "22feb46d048fdf2675eee7419b532d03"
server_key_iv: "7e10d35299c8768261e4a3e7e03aff4f"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:16.816427 9153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:16.845644 9153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:16.848615 9153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:16.848831 9153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:16.857266 9153 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:43607
I20260811 22:33:16.857286 9284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:43607 every 8 connection(s)
I20260811 22:33:16.858538 9153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:16.860885 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 9153
I20260811 22:33:16.861007 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:33:16.861361 19278 external_mini_cluster.cc:1428] Setting key 08d49e472ea5f50c5fc4cd6bb1790729
I20260811 22:33:16.862352 9285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:16.868741 9285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Bootstrap starting.
I20260811 22:33:16.871323 9285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:16.872164 9285 log.cc:824] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:33:16 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:16.874655 9285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: No bootstrap required, opened a new log
I20260811 22:33:16.879230 9285 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:16.879531 9285 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:16.879621 9285 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a08fb70ecb4aaf993be032843b759f, State: Initialized, Role: FOLLOWER
I20260811 22:33:16.879745 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:16.862835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.18.211.190:43607)
0811 22:33:16.863319 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:16.863340 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:16.865652 (+ 2312us) server_negotiation.cc:407] Connection header received
0811 22:33:16.866705 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:16.866735 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:16.867192 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:16.867565 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:16.868512 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:16.870040 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:16.871245 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:16.871978 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:16.872139 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:16.874922 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:16.874927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:16.874941 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:16.874969 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:16.876684 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:16.877159 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:16.877165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:16.877170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:16.877338 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:16.877660 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:16.877664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:16.877666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:16.878086 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:16.878313 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:16.878579 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 22:33:16.878915 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":152,"threads_started":1}
I20260811 22:33:16.880187 9285 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:16.880354 9285 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:33:16.880446 9285 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:33:16.880556 9285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:16.881925 9285 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:16.882372 9285 leader_election.cc:302] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1a08fb70ecb4aaf993be032843b759f; no voters:
I20260811 22:33:16.882745 9285 leader_election.cc:288] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:33:16.882922 9290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:16.883353 9290 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 LEADER]: Becoming Leader. State: Replica: f1a08fb70ecb4aaf993be032843b759f, State: Running, Role: LEADER
I20260811 22:33:16.883987 9290 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:16.884171 9285 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:16.888151 9290 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1a08fb70ecb4aaf993be032843b759f. Latest consensus state: current_term: 1 leader_uuid: "f1a08fb70ecb4aaf993be032843b759f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } } }
I20260811 22:33:16.888389 9290 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:16.888922 9285 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:33:16.889297 9317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:16.889639 9292 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1a08fb70ecb4aaf993be032843b759f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } } }
I20260811 22:33:16.889870 9292 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:16.894028 9317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:16.905498 9317 catalog_manager.cc:1347] Generated new cluster ID: dda152349e3e49378197b956b492549f
I20260811 22:33:16.905594 9317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:33:16.935434 9317 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:33:16.936864 9317 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:33:16.950125 9317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Generated new TSK 0
I20260811 22:33:16.951099 9317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:33:17.660051 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43607
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:17.808383 9360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:17.808738 9360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:17.808807 9360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:17.814777 9360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:17.814863 9360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:17.814950 9360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:33:17.821024 9360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.211.148:42693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:17.822573 9360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:17.823901 9360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:17.836122 9378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:17.836123 9381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:17.836604 9376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:17.837364 9360 server_base.cc:1034] running on GCE node
I20260811 22:33:17.837823 9360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:17.838497 9360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:17.839700 9360 hybrid_clock.cc:648] HybridClock initialized: now 1786487597839677 us; error 42 us; skew 500 ppm
Aug 11 22:33:17 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487597, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:17.842911 9360 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:33:17.844015 9360 webserver.cc:460] Webserver started at http://127.18.211.129:40195/ using document root <none> and password file <none>
I20260811 22:33:17.844748 9360 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:17.844833 9360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:17.845069 9360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:33:17 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487597, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:17.864583 9360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "a16cc7d7ec354883a48d71d0d0bd40c7"
format_stamp: "Formatted at 2026-08-11 22:33:17 on dist-test-slave-8hhm"
server_key: "626217c10596eb6abdd27765d11a8be2"
server_key_iv: "2707a3e753bf1355beb4d376505a40a8"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:17.868814 9360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "a16cc7d7ec354883a48d71d0d0bd40c7"
format_stamp: "Formatted at 2026-08-11 22:33:17 on dist-test-slave-8hhm"
server_key: "626217c10596eb6abdd27765d11a8be2"
server_key_iv: "2707a3e753bf1355beb4d376505a40a8"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:17.874281 9360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260811 22:33:17.893272 9394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:17.895042 9360 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260811 22:33:17.895246 9360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "a16cc7d7ec354883a48d71d0d0bd40c7"
format_stamp: "Formatted at 2026-08-11 22:33:17 on dist-test-slave-8hhm"
server_key: "626217c10596eb6abdd27765d11a8be2"
server_key_iv: "2707a3e753bf1355beb4d376505a40a8"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:17.895491 9360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:17.923465 9360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:17.927297 9360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:17.927579 9360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:17.928377 9360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:17.929559 9360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:17.929669 9360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:17.929800 9360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:17.929888 9360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:17.943835 9360 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:38765
I20260811 22:33:17.946527 9585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:38765 every 8 connection(s)
I20260811 22:33:17.946869 9360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:33:17.950438 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 9360
I20260811 22:33:17.950562 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:33:17.950870 19278 external_mini_cluster.cc:1428] Setting key 48483deb2fbcc14097f85d4ffb30a1c8
Aug 11 22:33:17 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487597, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:17.975786 9594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:17.950440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43607 (local address 127.18.211.129:45901)
0811 22:33:17.957690 (+ 7250us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:17.957750 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:17.958821 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:17.959619 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:17.959630 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:17.960043 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:17.960786 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:17.960810 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:17.962082 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:17.962087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:17.963583 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:17.963595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:17.964383 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:17.964387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:17.964586 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:17.965335 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:17.965357 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:17.967215 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:17.969675 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:17.969681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:17.969698 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:17.970076 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:17.970411 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:17.970414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:17.970415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:17.970616 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:17.973562 (+ 2946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:17.973571 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:17.973947 (+ 376us) client_negotiation.cc:769] Sending connection context
0811 22:33:17.974214 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 22:33:17.974601 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7026,"thread_start_us":147,"threads_started":1}
I20260811 22:33:17.976613 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:17.949538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:45901 (local address 127.18.211.190:43607)
0811 22:33:17.957404 (+ 7866us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:17.957412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:17.957872 (+ 460us) server_negotiation.cc:407] Connection header received
0811 22:33:17.959082 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:17.959086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:17.959145 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:17.959253 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:17.960994 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:17.961901 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:17.963724 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:17.964096 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:17.964193 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:17.967413 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:17.967416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:17.967419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:17.967446 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:17.969524 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:17.970203 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:17.970207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:17.970209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:17.970304 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:17.970741 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:17.970744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:17.970746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:17.970999 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:17.971095 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:17.976281 (+ 5186us) server_negotiation.cc:299] Negotiation successful
0811 22:33:17.976433 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7780,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:33:17.984091 9586 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43607
I20260811 22:33:17.984378 9586 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:17.985052 9586 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:17.987111 9218 ts_manager.cc:194] Registered new tserver with Master: a16cc7d7ec354883a48d71d0d0bd40c7 (127.18.211.129:38765)
I20260811 22:33:17.988690 9218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:45901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Aug 11 22:33:17 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487597, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:33:18.015053 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43607
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:18.183099 9600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:18.183565 9600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:18.183702 9600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:18.190018 9600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:18.190161 9600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:18.190317 9600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:33:18.198542 9600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.211.148:42693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:18.200124 9600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:18.201582 9600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:18.214396 9610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:18.214715 9600 server_base.cc:1034] running on GCE node
W20260811 22:33:18.214499 9614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:18.214920 9611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:18.215446 9600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:18.216284 9600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:18.217540 9600 hybrid_clock.cc:648] HybridClock initialized: now 1786487598217516 us; error 36 us; skew 500 ppm
Aug 11 22:33:18 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487598, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:18.221490 9600 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:33:18.222689 9600 webserver.cc:460] Webserver started at http://127.18.211.130:40365/ using document root <none> and password file <none>
I20260811 22:33:18.223443 9600 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:18.223546 9600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:18.223867 9600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:33:18 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487598, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:18.249012 9600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "d4365ca39b7f4b879fdb994007995fad"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "bd3c3927fd36603755a5e94cd4a6cf8c"
server_key_iv: "49f91fc77e54b04ed022d525a4f80ddc"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:18.250006 9600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "d4365ca39b7f4b879fdb994007995fad"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "bd3c3927fd36603755a5e94cd4a6cf8c"
server_key_iv: "49f91fc77e54b04ed022d525a4f80ddc"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:18.262837 9600 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.001s sys 0.004s
I20260811 22:33:18.285523 9629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:18.287294 9600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:33:18.287467 9600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "d4365ca39b7f4b879fdb994007995fad"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "bd3c3927fd36603755a5e94cd4a6cf8c"
server_key_iv: "49f91fc77e54b04ed022d525a4f80ddc"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:18.287642 9600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:18.333178 9600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:18.337296 9600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:18.337500 9600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:18.338265 9600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:18.339373 9600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:18.339438 9600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:18.339495 9600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:18.339529 9600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:18.353626 9600 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:34175
I20260811 22:33:18.354971 9600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:33:18.360390 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 9600
I20260811 22:33:18.360544 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:33:18.360847 19278 external_mini_cluster.cc:1428] Setting key 9716130dd71c4a1d7f8fc366fe8ce5a6
Aug 11 22:33:18 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487598, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:18.370055 9817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:34175 every 8 connection(s)
I20260811 22:33:18.373140 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:18.359273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:58933 (local address 127.18.211.190:43607)
0811 22:33:18.359432 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:18.359438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:18.359460 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:33:18.360123 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:18.360129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:18.360189 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:18.360253 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:18.362102 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.362836 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.364642 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.365029 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.365110 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:18.368379 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:18.368383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:18.368385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:18.368413 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:18.370330 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:18.371026 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:18.371032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:18.371037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:18.371134 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:18.371590 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:18.371593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:18.371594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:18.371884 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:18.371964 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:18.372723 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 22:33:18.372880 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:33:18.375309 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:18.358294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43607 (local address 127.18.211.130:58933)
0811 22:33:18.358791 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:18.358844 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:18.359864 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:18.360720 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:18.360730 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:18.361085 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:18.361914 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:18.361937 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.363007 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.363011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:18.364477 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:18.364493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.365333 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.365337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:18.365536 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:18.366341 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:18.366362 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:18.368176 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:18.370490 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:18.370498 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:18.370518 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:18.370901 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:18.371256 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:18.371260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:18.371263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:18.371458 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:18.372059 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:18.372067 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:18.372411 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 22:33:18.373740 (+ 1329us) client_negotiation.cc:240] Negotiation successful
0811 22:33:18.374148 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":133,"threads_started":1}
I20260811 22:33:18.376941 9818 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43607
I20260811 22:33:18.377239 9818 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:18.377838 9818 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:18.379026 9218 ts_manager.cc:194] Registered new tserver with Master: d4365ca39b7f4b879fdb994007995fad (127.18.211.130:34175)
I20260811 22:33:18.379772 9218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:58933
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:33:18.412591 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43607
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:33:18 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487597, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
W20260811 22:33:18.585915 9825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:18.586334 9825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:18.586438 9825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:18.592587 9825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:18.592787 9825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:18.592957 9825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:33:18.600513 9825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.211.148:42693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:18.602200 9825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:18.603909 9825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:18.617864 9837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:18.618908 9840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:18.620203 9836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:18.621882 9825 server_base.cc:1034] running on GCE node
I20260811 22:33:18.622550 9825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:18.623409 9825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:18.638942 9825 hybrid_clock.cc:648] HybridClock initialized: now 1786487598638846 us; error 103 us; skew 500 ppm
Aug 11 22:33:18 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487598, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:18.643218 9825 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:33:18.644435 9825 webserver.cc:460] Webserver started at http://127.18.211.131:36157/ using document root <none> and password file <none>
I20260811 22:33:18.645291 9825 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:18.645434 9825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:18.645732 9825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:33:18 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487598, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:18.667068 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "eaaeb1f745fa4710ba4ac60d40737bff"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "6e8d39760029ddde211c0890e462d8f2"
server_key_iv: "c2752a0c77cb1feb7f32419a6627d8f9"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:18.675414 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "eaaeb1f745fa4710ba4ac60d40737bff"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "6e8d39760029ddde211c0890e462d8f2"
server_key_iv: "c2752a0c77cb1feb7f32419a6627d8f9"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:18.693359 9825 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.005s sys 0.000s
I20260811 22:33:18.721638 9853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:18.723608 9825 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260811 22:33:18.723783 9825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "eaaeb1f745fa4710ba4ac60d40737bff"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "6e8d39760029ddde211c0890e462d8f2"
server_key_iv: "c2752a0c77cb1feb7f32419a6627d8f9"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:18.723946 9825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:18.747852 9825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:18.751652 9825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:18.751963 9825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:18.752712 9825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:18.753870 9825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:18.753993 9825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:18.754101 9825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:18.754169 9825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:18.779520 9825 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:40177
I20260811 22:33:18.780985 9825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:33:18.785424 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 9825
I20260811 22:33:18.785553 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:33:18.785868 19278 external_mini_cluster.cc:1428] Setting key 44a7135c2a03f7f40b3622bace48f2d8
I20260811 22:33:18.781909 10032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:40177 every 8 connection(s)
Aug 11 22:33:18 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487598, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:18.821461 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:18.792941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:57907 (local address 127.18.211.190:43607)
0811 22:33:18.793071 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:18.793076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:18.801116 (+ 8040us) server_negotiation.cc:407] Connection header received
0811 22:33:18.802730 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:18.802733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:18.802786 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:18.802833 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:18.804666 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.805508 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.810468 (+ 4960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.810842 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.810923 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:18.814170 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:18.814175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:18.814178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:18.814206 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:18.817134 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:18.817781 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:18.817785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:18.817787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:18.817875 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:18.819965 (+ 2090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:18.819968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:18.819971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:18.820238 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:18.820322 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:18.821102 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 22:33:18.821269 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:33:18.825320 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:18.791747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43607 (local address 127.18.211.131:57907)
0811 22:33:18.800921 (+ 9174us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:18.800997 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:18.802458 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:18.803351 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:18.803361 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:18.803745 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:18.804487 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:18.804509 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.805691 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.805695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:18.807096 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:18.807108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:18.811154 (+ 4046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:18.811158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:18.811367 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:18.812134 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:18.812156 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:18.813955 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:18.817289 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:18.817295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:18.817313 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:18.817663 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:18.817975 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:18.817978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:18.817979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:18.818161 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:18.820409 (+ 2248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:18.820416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:18.820748 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 22:33:18.822075 (+ 1327us) client_negotiation.cc:240] Negotiation successful
0811 22:33:18.822440 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8851,"thread_start_us":172,"threads_started":1}
I20260811 22:33:18.826814 10033 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43607
I20260811 22:33:18.827071 10033 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:18.827649 10033 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:18.828794 9218 ts_manager.cc:194] Registered new tserver with Master: eaaeb1f745fa4710ba4ac60d40737bff (127.18.211.131:40177)
I20260811 22:33:18.829633 9218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:57907
I20260811 22:33:18.835641 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:33:18.836489 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 9360
I20260811 22:33:18.937158 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 9600
I20260811 22:33:18.948164 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 9825
I20260811 22:33:18.959648 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 9153
I20260811 22:33:18.986706 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43607
--webserver_interface=127.18.211.190
--webserver_port=42005
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:43607
--ranger_config_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:19.120915 10043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:19.121271 10043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:19.121372 10043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:19.125523 10043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:19.125608 10043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:19.125648 10043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:19.125687 10043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:19.125715 10043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:19.131284 10043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:43607
--ranger_config_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:19.132454 10043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:19.133675 10043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:19.142791 10055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:19.142831 10043 server_base.cc:1034] running on GCE node
W20260811 22:33:19.142762 10053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:19.142762 10058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:19.143592 10043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:19.144765 10043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:19.145949 10043 hybrid_clock.cc:648] HybridClock initialized: now 1786487599145931 us; error 36 us; skew 500 ppm
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:19.148665 10043 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:19.149588 10043 webserver.cc:460] Webserver started at http://127.18.211.190:42005/ using document root <none> and password file <none>
I20260811 22:33:19.150106 10043 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:19.150173 10043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:19.224792 10043 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:33:19.227490 10070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:19.228906 10043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:33:19.229060 10043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "f1a08fb70ecb4aaf993be032843b759f"
format_stamp: "Formatted at 2026-08-11 22:33:16 on dist-test-slave-8hhm"
server_key: "22feb46d048fdf2675eee7419b532d03"
server_key_iv: "7e10d35299c8768261e4a3e7e03aff4f"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:19.229723 10043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:19.246971 10043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:19.250072 10043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:19.250288 10043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:19.257869 10043 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:43607
I20260811 22:33:19.257879 10156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:43607 every 8 connection(s)
I20260811 22:33:19.258978 10043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:19.263450 10158 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:33:19.265089 10158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Bootstrap starting.
I20260811 22:33:19.266522 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 10043
I20260811 22:33:19.266971 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:38765
--local_ip_for_outbound_sockets=127.18.211.129
--tserver_master_addrs=127.18.211.190:43607
--webserver_port=40195
--webserver_interface=127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:19.282761 10158 log.cc:824] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:19.289757 10158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:33:19.290158 10158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Bootstrap complete.
I20260811 22:33:19.294519 10158 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:19.294927 10158 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a08fb70ecb4aaf993be032843b759f, State: Initialized, Role: FOLLOWER
I20260811 22:33:19.295439 10158 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:19.295557 10158 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:33:19.295642 10158 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:33:19.295740 10158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:33:19.297753 10158 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:19.298091 10158 leader_election.cc:302] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1a08fb70ecb4aaf993be032843b759f; no voters:
I20260811 22:33:19.298460 10158 leader_election.cc:288] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:33:19.298524 10165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:33:19.298827 10165 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [term 2 LEADER]: Becoming Leader. State: Replica: f1a08fb70ecb4aaf993be032843b759f, State: Running, Role: LEADER
I20260811 22:33:19.299155 10165 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } }
I20260811 22:33:19.299839 10158 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:19.300642 10167 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f1a08fb70ecb4aaf993be032843b759f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } } }
I20260811 22:33:19.300791 10167 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:19.301636 10166 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1a08fb70ecb4aaf993be032843b759f. Latest consensus state: current_term: 2 leader_uuid: "f1a08fb70ecb4aaf993be032843b759f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a08fb70ecb4aaf993be032843b759f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43607 } } }
I20260811 22:33:19.301743 10166 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:19.302049 10179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:19.302449 10158 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:33:19.308322 10179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:19.309029 10179 catalog_manager.cc:1259] Loaded cluster ID: dda152349e3e49378197b956b492549f
I20260811 22:33:19.309094 10179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:33:19.311792 10179 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:33:19.312456 10179 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Loaded TSK: 0
I20260811 22:33:19.313687 10179 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:33:19.420894 10161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:19.421319 10161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:19.421437 10161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:19.427697 10161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:19.427781 10161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:19.427868 10161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:33:19.435880 10161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:38765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=40195
--tserver_master_addrs=127.18.211.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:19.437345 10161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:19.438743 10161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:19.451813 10226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:19.452195 10161 server_base.cc:1034] running on GCE node
W20260811 22:33:19.452283 10225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:19.452201 10230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:19.452869 10161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:19.453629 10161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:19.454804 10161 hybrid_clock.cc:648] HybridClock initialized: now 1786487599454771 us; error 53 us; skew 500 ppm
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:19.458353 10161 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:33:19.459446 10161 webserver.cc:460] Webserver started at http://127.18.211.129:40195/ using document root <none> and password file <none>
I20260811 22:33:19.460175 10161 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:19.460268 10161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:19.480250 10161 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:33:19.483440 10244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:19.485091 10161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:33:19.485257 10161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "a16cc7d7ec354883a48d71d0d0bd40c7"
format_stamp: "Formatted at 2026-08-11 22:33:17 on dist-test-slave-8hhm"
server_key: "626217c10596eb6abdd27765d11a8be2"
server_key_iv: "2707a3e753bf1355beb4d376505a40a8"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:19.486037 10161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:19.518680 10161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:19.522636 10161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:19.522866 10161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:19.523653 10161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:19.524751 10161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:19.524820 10161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:19.524883 10161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:19.524910 10161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:19.536835 10161 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:38765
I20260811 22:33:19.536953 10436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:38765 every 8 connection(s)
I20260811 22:33:19.539804 10161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:33:19.546093 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 10161
I20260811 22:33:19.546646 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:34175
--local_ip_for_outbound_sockets=127.18.211.130
--tserver_master_addrs=127.18.211.190:43607
--webserver_port=40365
--webserver_interface=127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:19.560412 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:19.542358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43607 (local address 127.18.211.129:42501)
0811 22:33:19.543251 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:19.543317 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:19.544221 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:19.545609 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:19.545619 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:19.545960 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:19.546519 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:19.546534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.547885 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.547888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:19.549094 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:19.549108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.550550 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.550554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:19.550777 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:19.551561 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:19.551588 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:19.553586 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:19.556486 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:19.556495 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:19.556512 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:19.556899 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:19.557401 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:19.557404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:19.557406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:19.557583 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:19.558390 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:19.558399 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:19.558740 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 22:33:19.558955 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 22:33:19.559308 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":155,"threads_started":1}
I20260811 22:33:19.562301 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:19.542347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:42501 (local address 127.18.211.190:43607)
0811 22:33:19.542952 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:19.542975 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:19.543471 (+ 496us) server_negotiation.cc:407] Connection header received
0811 22:33:19.544710 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:19.544727 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:19.545099 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:19.545498 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:19.546687 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.547750 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.549281 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.550136 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.550308 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:19.553843 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:19.553851 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:19.553874 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:19.553915 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:19.556351 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:19.557060 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:19.557067 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:19.557073 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:19.557246 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:19.557717 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:19.557720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:19.557722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:19.558194 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:19.558318 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:19.560986 (+ 2668us) server_negotiation.cc:299] Negotiation successful
0811 22:33:19.561325 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":124,"threads_started":1}
I20260811 22:33:19.566325 10438 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43607
I20260811 22:33:19.566613 10438 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:19.567291 10438 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
W20260811 22:33:19.569677 10438 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:43607 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:33:19.569823 10438 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:19.570019 10438 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:19.570883 10438 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:19.571063 10438 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
W20260811 22:33:19.571844 10438 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:33:19.696960 10443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:19.697292 10443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:19.697373 10443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:19.702339 10443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:19.702425 10443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:19.702531 10443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:33:19.708757 10443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:34175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=40365
--tserver_master_addrs=127.18.211.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:19.710042 10443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:19.711215 10443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:19.723377 10453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:19.723892 10443 server_base.cc:1034] running on GCE node
W20260811 22:33:19.723644 10456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:19.724046 10451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:19.725824 10443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:19.726830 10443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:19.730156 10443 hybrid_clock.cc:648] HybridClock initialized: now 1786487599730114 us; error 47 us; skew 500 ppm
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:19.738255 10443 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:33:19.739355 10443 webserver.cc:460] Webserver started at http://127.18.211.130:40365/ using document root <none> and password file <none>
I20260811 22:33:19.739979 10443 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:19.740093 10443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:19.765869 10443 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.000s
I20260811 22:33:19.768739 10468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:19.770213 10443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:33:19.770318 10443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "d4365ca39b7f4b879fdb994007995fad"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "bd3c3927fd36603755a5e94cd4a6cf8c"
server_key_iv: "49f91fc77e54b04ed022d525a4f80ddc"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:19.771030 10443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:19.795368 10443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:19.799654 10443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:19.799952 10443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:19.800814 10443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:19.801954 10443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:19.802093 10443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:19.802206 10443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:19.802281 10443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:19.819769 10443 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:34175
I20260811 22:33:19.821244 10443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:33:19.821869 10659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:34175 every 8 connection(s)
I20260811 22:33:19.829205 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 10443
I20260811 22:33:19.829763 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.211.148:42693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:40177
--local_ip_for_outbound_sockets=127.18.211.131
--tserver_master_addrs=127.18.211.190:43607
--webserver_port=36157
--webserver_interface=127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:33:19 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:19.843309 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:19.824330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:50003 (local address 127.18.211.190:43607)
0811 22:33:19.824495 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:19.824500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:19.825748 (+ 1248us) server_negotiation.cc:407] Connection header received
0811 22:33:19.827133 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:19.827138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:19.827210 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:19.827325 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:19.828731 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.829680 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.832407 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.832799 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.832908 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:19.837859 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:19.837864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:19.837868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:19.837904 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:19.840149 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:19.841011 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:19.841016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:19.841018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:19.841128 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:19.841667 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:19.841671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:19.841673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:19.841941 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:19.842059 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:19.842885 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 22:33:19.843092 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:33:19.845402 10662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:19.824925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43607 (local address 127.18.211.130:50003)
0811 22:33:19.825561 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:19.825621 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:19.826820 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:19.827532 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:19.827541 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:19.827905 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:19.828544 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:19.828562 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.829881 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.829885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:19.832217 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:19.832232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:19.834260 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:19.834265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:19.834511 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:19.835384 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:19.835412 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:19.837599 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:19.840326 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:19.840335 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:19.840358 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:19.840823 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:19.841307 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:19.841311 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:19.841313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:19.841527 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:19.842165 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:19.842174 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:19.842544 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 22:33:19.843659 (+ 1115us) client_negotiation.cc:240] Negotiation successful
0811 22:33:19.844070 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":212,"threads_started":1}
I20260811 22:33:19.846920 10660 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43607
I20260811 22:33:19.847234 10660 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:19.847931 10660 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
W20260811 22:33:19.850018 10660 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:43607 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:33:19.850171 10660 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:19.850379 10660 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:19.851213 10660 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:19.851378 10660 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
W20260811 22:33:19.852139 10660 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:33:20.000605 10663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:20.001093 10663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:20.001252 10663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:20.007954 10663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:20.008046 10663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:20.008139 10663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:33:20.016577 10663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.211.148:42693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:40177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=36157
--tserver_master_addrs=127.18.211.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:20.018088 10663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:20.019522 10663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:20.032511 10674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:20.032590 10675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:20.032647 10678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:20.033581 10663 server_base.cc:1034] running on GCE node
I20260811 22:33:20.034093 10663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:20.034979 10663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:20.036196 10663 hybrid_clock.cc:648] HybridClock initialized: now 1786487600036168 us; error 39 us; skew 500 ppm
Aug 11 22:33:20 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:20.039953 10663 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:33:20.041159 10663 webserver.cc:460] Webserver started at http://127.18.211.131:36157/ using document root <none> and password file <none>
I20260811 22:33:20.041988 10663 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:20.042102 10663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:33:20 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:20.061892 10663 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:33:20.065421 10698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:20.066999 10663 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:33:20.067238 10663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "eaaeb1f745fa4710ba4ac60d40737bff"
format_stamp: "Formatted at 2026-08-11 22:33:18 on dist-test-slave-8hhm"
server_key: "6e8d39760029ddde211c0890e462d8f2"
server_key_iv: "c2752a0c77cb1feb7f32419a6627d8f9"
server_key_version: "kuduclusterkey@0"
I20260811 22:33:20.068104 10663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:20.099970 10663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:20.103509 10663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:20.103730 10663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:20.104399 10663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:20.105497 10663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:20.105564 10663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:20.105618 10663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:20.105646 10663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:20.117054 10663 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:40177
I20260811 22:33:20.117115 10890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:40177 every 8 connection(s)
I20260811 22:33:20.118397 10663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:33:20.123314 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 10663
Aug 11 22:33:20 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:20.135007 10906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.124022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.18.211.190:43607)
0811 22:33:20.124940 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:20.124951 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:20.124962 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:33:20.124991 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:20.124994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:20.125035 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:20.125111 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:20.126195 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.126970 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.128065 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.128422 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.128521 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.130277 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:20.130281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:20.130284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:20.130308 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:20.132326 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:20.132922 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:20.132925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:20.132927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:20.133041 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:20.133863 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:20.133866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:20.133868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:20.134087 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:20.134330 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:20.134675 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 22:33:20.134812 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":107,"threads_started":1}
I20260811 22:33:20.135629 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.120535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:40973 (local address 127.18.211.190:43607)
0811 22:33:20.120667 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:20.120670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:20.121775 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 22:33:20.122749 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:20.122753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:20.122807 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:20.122890 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:20.124963 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.125950 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.127652 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.128027 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.128150 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.131363 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:20.131366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:20.131370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:20.131420 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:20.133254 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:20.133828 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:20.133836 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:20.133837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:20.133908 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:20.134340 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:20.134345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:20.134348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:20.134580 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:20.134668 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:20.135378 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 22:33:20.135495 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:33:20.137137 10903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.121082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43607 (local address 127.18.211.131:40973)
0811 22:33:20.121622 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:20.121666 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:20.122540 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:20.123083 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:20.123093 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:20.123469 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:20.124795 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.124820 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.126128 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.126133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.127493 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.127504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.128188 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.128192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.128389 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.129521 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:20.129536 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:20.131211 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:20.133363 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:20.133369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:20.133386 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:20.133733 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:20.134004 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:20.134008 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:20.134010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:20.134210 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:20.134730 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:20.134739 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:20.135069 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 22:33:20.135792 (+ 723us) client_negotiation.cc:240] Negotiation successful
0811 22:33:20.136142 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":158,"threads_started":1}
I20260811 22:33:20.138460 10891 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43607
I20260811 22:33:20.138736 10891 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:20.139345 10891 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:20.140810 10095 ts_manager.cc:194] Registered new tserver with Master: eaaeb1f745fa4710ba4ac60d40737bff (127.18.211.131:40177)
I20260811 22:33:20.142005 10095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:40973
Aug 11 22:33:20 dist-test-slave-8hhm krb5kdc[8367](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:20.572070 10438 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:20.572402 10438 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:20.573175 10095 ts_manager.cc:194] Registered new tserver with Master: a16cc7d7ec354883a48d71d0d0bd40c7 (127.18.211.129:38765)
I20260811 22:33:20.573864 10095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:42501
I20260811 22:33:20.852403 10660 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:20.852759 10660 heartbeater.cc:511] Master 127.18.211.190:43607 requested a full tablet report, sending...
I20260811 22:33:20.853556 10095 ts_manager.cc:194] Registered new tserver with Master: d4365ca39b7f4b879fdb994007995fad (127.18.211.130:34175)
I20260811 22:33:20.854221 10095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:50003
I20260811 22:33:20.860625 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:33:20.871309 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.862442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.18.211.190:43607)
0811 22:33:20.862980 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:20.862985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:20.863087 (+ 102us) server_negotiation.cc:407] Connection header received
0811 22:33:20.863262 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:20.863265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:20.863319 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:20.863426 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:20.864478 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.865032 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.865923 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.866167 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.866292 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.867851 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:20.867855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:20.867858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:20.867883 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:20.869497 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:20.869970 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:20.869974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:20.869976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:20.870063 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:20.870345 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:20.870348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:20.870350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:20.870516 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:20.870651 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:20.870879 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 22:33:20.871020 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":81,"threads_started":1}
I20260811 22:33:20.877485 10095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:33:20.881704 10095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:33:20.909148 10945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.895155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38765 (local address 127.0.0.1:43274)
0811 22:33:20.897738 (+ 2583us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:20.897792 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:20.898701 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:20.899479 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:20.899486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:20.899511 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:20.899642 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.899649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.901320 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.901326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.906645 (+ 5319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.906656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.908515 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.908519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.908657 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.908708 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:33:20.908829 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:33:20.908973 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2391,"mutex_wait_us":3105,"thread_start_us":103,"threads_started":1}
I20260811 22:33:20.910220 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.896527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.18.211.129:38765)
0811 22:33:20.897304 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:20.897312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:20.897920 (+ 608us) server_negotiation.cc:407] Connection header received
0811 22:33:20.898924 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:20.898931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:20.899162 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:20.899297 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:20.899795 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.901137 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.906789 (+ 5652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.907623 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.907759 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.907897 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:20.909943 (+ 2046us) server_negotiation.cc:299] Negotiation successful
0811 22:33:20.910057 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":111,"threads_started":1}
I20260811 22:33:20.913864 10332 tablet_service.cc:1467] Processing CreateTablet for tablet 15f27226805944cf91aa6901bfdb3443 (DEFAULT_TABLE table=test-table [id=529c68a9a31a49d382d491dd5785ade4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:20.915241 10332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f27226805944cf91aa6901bfdb3443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:20.909428 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.895228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:34175 (local address 127.0.0.1:52968)
0811 22:33:20.900089 (+ 4861us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:20.900103 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:20.900184 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:20.901656 (+ 1472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:20.901659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:20.901676 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:20.901771 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.901777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.903452 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.903455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.904735 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.904745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.908693 (+ 3948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.908697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.908812 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.908834 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:33:20.908885 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 22:33:20.909299 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4801,"mutex_wait_us":334,"thread_start_us":108,"threads_started":1}
I20260811 22:33:20.921739 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.896864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.18.211.130:34175)
0811 22:33:20.897461 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:20.897466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:20.900278 (+ 2812us) server_negotiation.cc:407] Connection header received
0811 22:33:20.900361 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:20.900366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:20.900543 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:20.900673 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:20.901896 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.903109 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.905822 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.906869 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.907036 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.907184 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:20.909004 (+ 1820us) server_negotiation.cc:299] Negotiation successful
0811 22:33:20.909117 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":130,"threads_started":1}
I20260811 22:33:20.922449 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.896254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.18.211.131:40177)
0811 22:33:20.899930 (+ 3676us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:20.899937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:20.901095 (+ 1158us) server_negotiation.cc:407] Connection header received
0811 22:33:20.901175 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:20.901180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:20.901392 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:20.901518 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:20.905405 (+ 3887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.906466 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.909541 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.910372 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.910495 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.922103 (+ 11608us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:20.922169 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 22:33:20.922289 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3570,"thread_start_us":103,"threads_started":1}
I20260811 22:33:20.924902 10546 tablet_service.cc:1467] Processing CreateTablet for tablet 15f27226805944cf91aa6901bfdb3443 (DEFAULT_TABLE table=test-table [id=529c68a9a31a49d382d491dd5785ade4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:20.926074 10546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f27226805944cf91aa6901bfdb3443. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:33:20 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:20.936396 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:20.896021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:40177 (local address 127.0.0.1:43202)
0811 22:33:20.900891 (+ 4870us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:20.900905 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:20.900988 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:20.903240 (+ 2252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:20.903243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:20.903263 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:20.903349 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.903355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.907249 (+ 3894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.907253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.908551 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:20.908563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:20.921297 (+ 12734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:20.921301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:20.921425 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:20.921665 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 22:33:20.921745 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:33:20.936241 (+ 14496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4809,"thread_start_us":82,"threads_started":1}
I20260811 22:33:20.939945 10770 tablet_service.cc:1467] Processing CreateTablet for tablet 15f27226805944cf91aa6901bfdb3443 (DEFAULT_TABLE table=test-table [id=529c68a9a31a49d382d491dd5785ade4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:20.940881 10955 tablet_bootstrap.cc:492] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: Bootstrap starting.
I20260811 22:33:20.941246 10770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f27226805944cf91aa6901bfdb3443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:20.943727 10955 tablet_bootstrap.cc:654] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:20.944176 10956 tablet_bootstrap.cc:492] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7: Bootstrap starting.
I20260811 22:33:20.947551 10956 tablet_bootstrap.cc:654] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:20.948094 10955 log.cc:824] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:20.949391 10956 log.cc:824] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:20.952147 10955 tablet_bootstrap.cc:492] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: No bootstrap required, opened a new log
I20260811 22:33:20.952446 10955 ts_tablet_manager.cc:1397] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260811 22:33:20.956494 10955 raft_consensus.cc:348] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:20.956892 10955 raft_consensus.cc:374] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:20.956998 10955 raft_consensus.cc:729] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4365ca39b7f4b879fdb994007995fad, State: Initialized, Role: FOLLOWER
I20260811 22:33:20.958616 10955 consensus_queue.cc:260] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:20.959832 10955 ts_tablet_manager.cc:1428] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:33:20.960348 10660 heartbeater.cc:503] Master 127.18.211.190:43607 was elected leader, sending a full tablet report...
I20260811 22:33:20.966820 10956 tablet_bootstrap.cc:492] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7: No bootstrap required, opened a new log
I20260811 22:33:20.967085 10956 ts_tablet_manager.cc:1397] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.003s
I20260811 22:33:20.971223 10956 raft_consensus.cc:348] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:20.971613 10956 raft_consensus.cc:374] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:20.971726 10956 raft_consensus.cc:729] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16cc7d7ec354883a48d71d0d0bd40c7, State: Initialized, Role: FOLLOWER
I20260811 22:33:20.972321 10956 consensus_queue.cc:260] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:20.974586 10956 ts_tablet_manager.cc:1428] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:33:20.975016 10438 heartbeater.cc:503] Master 127.18.211.190:43607 was elected leader, sending a full tablet report...
I20260811 22:33:20.979421 10961 tablet_bootstrap.cc:492] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Bootstrap starting.
I20260811 22:33:20.982089 10961 tablet_bootstrap.cc:654] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:20.983067 10961 log.cc:824] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:20.985581 10961 tablet_bootstrap.cc:492] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: No bootstrap required, opened a new log
I20260811 22:33:20.985785 10961 ts_tablet_manager.cc:1397] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260811 22:33:20.990477 10961 raft_consensus.cc:348] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:20.990770 10961 raft_consensus.cc:374] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:20.990815 10961 raft_consensus.cc:729] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaaeb1f745fa4710ba4ac60d40737bff, State: Initialized, Role: FOLLOWER
I20260811 22:33:20.991458 10961 consensus_queue.cc:260] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:20.992396 10891 heartbeater.cc:503] Master 127.18.211.190:43607 was elected leader, sending a full tablet report...
I20260811 22:33:20.993536 10961 ts_tablet_manager.cc:1428] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
W20260811 22:33:21.042820 10439 tablet.cc:2369] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:33:21.073474 10661 tablet.cc:2369] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:33:21.120097 10892 tablet.cc:2369] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:33:21.148876 10962 raft_consensus.cc:482] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:21.149122 10962 raft_consensus.cc:504] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:21.150480 10962 leader_election.cc:288] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4365ca39b7f4b879fdb994007995fad (127.18.211.130:34175), eaaeb1f745fa4710ba4ac60d40737bff (127.18.211.131:40177)
I20260811 22:33:21.156452 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:21.152096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:46805 (local address 127.18.211.131:40177)
0811 22:33:21.152222 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:21.152226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:21.152356 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:33:21.152545 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:21.152548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:21.152594 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:21.152640 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:21.153011 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.153761 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.155200 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.155751 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.155842 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:21.155888 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:21.156249 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 22:33:21.156332 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:33:21.156454 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:21.151327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:54297 (local address 127.18.211.130:34175)
0811 22:33:21.151452 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:21.151456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:21.151475 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:33:21.151635 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:21.151639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:21.151693 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:21.151762 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:21.152543 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.153338 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.155080 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.155675 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.155772 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:21.155828 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:21.156217 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 22:33:21.156302 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:33:21.156997 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:21.150977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:34175 (local address 127.18.211.129:54297)
0811 22:33:21.151330 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:21.151348 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:21.151474 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:21.151933 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:21.151936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:21.151957 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:21.152392 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:21.152401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.153486 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.153489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:21.154918 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:21.154930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.155932 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.155936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:21.156058 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:21.156084 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:33:21.156756 (+ 672us) client_negotiation.cc:240] Negotiation successful
0811 22:33:21.156834 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":125,"threads_started":1}
I20260811 22:33:21.157372 10968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:21.151682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:40177 (local address 127.18.211.129:46805)
0811 22:33:21.152271 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:21.152284 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:21.152423 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:21.152761 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:21.152764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:21.152780 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:21.152887 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:21.152892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.153903 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.153906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:21.155055 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:21.155066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.155995 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.155999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:21.156106 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:21.156127 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:33:21.156862 (+ 735us) client_negotiation.cc:240] Negotiation successful
0811 22:33:21.156939 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":99,"threads_started":1}
I20260811 22:33:21.158061 10571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f27226805944cf91aa6901bfdb3443" candidate_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4365ca39b7f4b879fdb994007995fad" is_pre_election: true
I20260811 22:33:21.158331 10773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f27226805944cf91aa6901bfdb3443" candidate_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" is_pre_election: true
I20260811 22:33:21.158420 10571 raft_consensus.cc:2454] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16cc7d7ec354883a48d71d0d0bd40c7 in term 0.
I20260811 22:33:21.158602 10773 raft_consensus.cc:2454] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16cc7d7ec354883a48d71d0d0bd40c7 in term 0.
I20260811 22:33:21.158937 10245 leader_election.cc:302] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16cc7d7ec354883a48d71d0d0bd40c7, d4365ca39b7f4b879fdb994007995fad; no voters:
I20260811 22:33:21.159256 10962 raft_consensus.cc:2781] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:33:21.159353 10962 raft_consensus.cc:482] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:33:21.159394 10962 raft_consensus.cc:3037] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:21.160732 10962 raft_consensus.cc:504] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:21.161132 10962 leader_election.cc:288] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [CANDIDATE]: Term 1 election: Requested vote from peers d4365ca39b7f4b879fdb994007995fad (127.18.211.130:34175), eaaeb1f745fa4710ba4ac60d40737bff (127.18.211.131:40177)
I20260811 22:33:21.161592 10571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f27226805944cf91aa6901bfdb3443" candidate_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4365ca39b7f4b879fdb994007995fad"
I20260811 22:33:21.161616 10773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f27226805944cf91aa6901bfdb3443" candidate_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaaeb1f745fa4710ba4ac60d40737bff"
I20260811 22:33:21.161713 10571 raft_consensus.cc:3037] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:21.161729 10773 raft_consensus.cc:3037] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:21.163106 10571 raft_consensus.cc:2454] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16cc7d7ec354883a48d71d0d0bd40c7 in term 1.
I20260811 22:33:21.163139 10773 raft_consensus.cc:2454] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16cc7d7ec354883a48d71d0d0bd40c7 in term 1.
I20260811 22:33:21.163460 10248 leader_election.cc:302] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16cc7d7ec354883a48d71d0d0bd40c7, eaaeb1f745fa4710ba4ac60d40737bff; no voters:
I20260811 22:33:21.163640 10962 raft_consensus.cc:2781] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:21.163849 10962 raft_consensus.cc:686] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [term 1 LEADER]: Becoming Leader. State: Replica: a16cc7d7ec354883a48d71d0d0bd40c7, State: Running, Role: LEADER
I20260811 22:33:21.164212 10962 consensus_queue.cc:237] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } }
I20260811 22:33:21.167404 10095 catalog_manager.cc:5495] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a16cc7d7ec354883a48d71d0d0bd40c7 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16cc7d7ec354883a48d71d0d0bd40c7" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 38765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:33:21 dist-test-slave-8hhm krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for HTTP/127.18.211.148@KRBTEST.COM
I20260811 22:33:21.203269 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:21.198055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.18.211.129:38765)
0811 22:33:21.198193 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:21.198198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:21.198210 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:33:21.198296 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:21.198298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:21.198353 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:21.198469 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:33:21.198746 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.199557 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.200936 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:21.201331 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:21.201432 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:21.201894 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:33:21.202514 (+ 620us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:33:21.202647 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:21.203017 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 22:33:21.203102 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:33:21.217149 10773 raft_consensus.cc:1261] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 1 FOLLOWER]: Refusing update from remote peer a16cc7d7ec354883a48d71d0d0bd40c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:21.217464 10571 raft_consensus.cc:1261] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 1 FOLLOWER]: Refusing update from remote peer a16cc7d7ec354883a48d71d0d0bd40c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:21.217950 10962 consensus_queue.cc:1035] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaaeb1f745fa4710ba4ac60d40737bff" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:21.218195 10971 consensus_queue.cc:1035] T 15f27226805944cf91aa6901bfdb3443 P a16cc7d7ec354883a48d71d0d0bd40c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4365ca39b7f4b879fdb994007995fad" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 34175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:21.233803 10980 mvcc.cc:205] Tried to move back new op lower bound from 7317453214575063040 to 7317453214371553280. Current Snapshot: MvccSnapshot[applied={T|T < 7317453214575063040}]
I20260811 22:33:21.235124 10978 mvcc.cc:205] Tried to move back new op lower bound from 7317453214575063040 to 7317453214371553280. Current Snapshot: MvccSnapshot[applied={T|T < 7317453214575063040}]
I20260811 22:33:21.249341 10094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:21.249536 10094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:21.253324 10094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1a08fb70ecb4aaf993be032843b759f: Sending DeleteTablet for 3 replicas of tablet 15f27226805944cf91aa6901bfdb3443
I20260811 22:33:21.254549 10332 tablet_service.cc:1514] Processing DeleteTablet for tablet 15f27226805944cf91aa6901bfdb3443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:21 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:43274
I20260811 22:33:21.255111 10770 tablet_service.cc:1514] Processing DeleteTablet for tablet 15f27226805944cf91aa6901bfdb3443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:21 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:43202
I20260811 22:33:21.256265 10546 tablet_service.cc:1514] Processing DeleteTablet for tablet 15f27226805944cf91aa6901bfdb3443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:21 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:52968
I20260811 22:33:21.257552 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 10161
I20260811 22:33:21.264081 10982 mvcc.cc:205] Tried to move back new op lower bound from 7317453214575063040 to 7317453214371553280. Current Snapshot: MvccSnapshot[applied={T|T < 7317453214575063040}]
I20260811 22:33:21.269428 10993 tablet_replica.cc:330] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: stopping tablet replica
I20260811 22:33:21.269717 10994 tablet_replica.cc:330] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: stopping tablet replica
I20260811 22:33:21.270166 10993 raft_consensus.cc:2229] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:21.270673 10994 raft_consensus.cc:2229] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:21.271273 10994 raft_consensus.cc:2258] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:21.271625 10993 raft_consensus.cc:2258] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:21.274282 10994 ts_tablet_manager.cc:1905] T 15f27226805944cf91aa6901bfdb3443 P d4365ca39b7f4b879fdb994007995fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:21.275383 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 10443
W20260811 22:33:21.275660 10076 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:38765 (error 108)
I20260811 22:33:21.280869 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:21.280524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:38765 (local address 127.0.0.1:43292)
0811 22:33:21.280707 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:21.280777 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:38765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260811 22:33:21.281060 10076 catalog_manager.cc:4584] TS a16cc7d7ec354883a48d71d0d0bd40c7 (127.18.211.129:38765): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15f27226805944cf91aa6901bfdb3443: Network error: Client connection negotiation failed: client connection to 127.18.211.129:38765: connect: Connection refused (error 111)
I20260811 22:33:21.298027 10993 ts_tablet_manager.cc:1905] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:21.301113 10993 ts_tablet_manager.cc:1918] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 22:33:21.302152 10947 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:33:21.301144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:34175 (local address 127.0.0.1:52978)
0811 22:33:21.301367 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:21.301383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:21.301491 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:21.302035 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.130:34175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":142}
I20260811 22:33:21.303120 10993 log.cc:1196] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/15f27226805944cf91aa6901bfdb3443
I20260811 22:33:21.303747 10993 ts_tablet_manager.cc:1939] T 15f27226805944cf91aa6901bfdb3443 P eaaeb1f745fa4710ba4ac60d40737bff: Deleting consensus metadata
I20260811 22:33:21.304991 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 10663
I20260811 22:33:21.328339 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:21.328042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:40177 (local address 127.0.0.1:43208)
0811 22:33:21.328159 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:21.328229 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.131:40177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 22:33:21.328363 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 10043
I20260811 22:33:21.354280 19278 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:33:21.709473 19278 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:33:21.709596 19278 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:33:22.054776 19278 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:33:22.054 UTC [8439] LOG: received smart shutdown request
2026-08-11 22:33:22.059 UTC [8439] LOG: background worker "logical replication launcher" (PID 8459) exited with exit code 1
2026-08-11 22:33:22.059 UTC [8451] LOG: shutting down
2026-08-11 22:33:22.220 UTC [8439] LOG: database system is shut down
2026-08-11T22:33:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50515 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:33:22 dist-test-slave-8hhm krb5kdc[11010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:33:22 dist-test-slave-8hhm krb5kdc[11010](info): set up 2 sockets
Aug 11 22:33:22 dist-test-slave-8hhm krb5kdc[11010](info): commencing operation
krb5kdc: starting...
W20260811 22:33:23.654091 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:33:23 dist-test-slave-8hhm krb5kdc[11010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:33:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:33:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:33:23.782505 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45915
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45915,127.18.211.189:43965,127.18.211.188:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:23.896102 11032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:23.896410 11032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:23.896488 11032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:23.900488 11032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:23.900568 11032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:23.900605 11032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:23.900655 11032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:23.900695 11032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:23.905927 11032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45915,127.18.211.189:43965,127.18.211.188:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:23.906985 11032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:23.908003 11032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:23.916466 11049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:23.916536 11053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:23.916592 11032 server_base.cc:1034] running on GCE node
W20260811 22:33:23.916466 11047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:23.917377 11032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:23.920887 11032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:23.922079 11032 hybrid_clock.cc:648] HybridClock initialized: now 1786487603922039 us; error 60 us; skew 500 ppm
Aug 11 22:33:23 dist-test-slave-8hhm krb5kdc[11010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:23.924986 11032 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:23.925951 11032 webserver.cc:460] Webserver started at http://127.18.211.190:44603/ using document root <none> and password file <none>
I20260811 22:33:23.926525 11032 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:23.926618 11032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:23.926860 11032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:23.929030 11032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "acbb7875964d4e9bb61b6ccf5c0c3273"
format_stamp: "Formatted at 2026-08-11 22:33:23 on dist-test-slave-8hhm"
server_key: "405f287f5c6d9e87294fae8a76d4b2f3"
server_key_iv: "8b29b0a306f6ab12e248113081caa797"
server_key_version: "encryptionkey@0"
I20260811 22:33:23.929674 11032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "acbb7875964d4e9bb61b6ccf5c0c3273"
format_stamp: "Formatted at 2026-08-11 22:33:23 on dist-test-slave-8hhm"
server_key: "405f287f5c6d9e87294fae8a76d4b2f3"
server_key_iv: "8b29b0a306f6ab12e248113081caa797"
server_key_version: "encryptionkey@0"
I20260811 22:33:23.933454 11032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:33:23.936002 11061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:23.937323 11032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:33:23.937458 11032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "acbb7875964d4e9bb61b6ccf5c0c3273"
format_stamp: "Formatted at 2026-08-11 22:33:23 on dist-test-slave-8hhm"
server_key: "405f287f5c6d9e87294fae8a76d4b2f3"
server_key_iv: "8b29b0a306f6ab12e248113081caa797"
server_key_version: "encryptionkey@0"
I20260811 22:33:23.937570 11032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:23.973668 11032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:23.977155 11032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:23.977396 11032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:23.984612 11032 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45915
I20260811 22:33:23.984899 11157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45915 every 8 connection(s)
I20260811 22:33:23.985646 11032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:23.988420 11159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:23.989487 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 11032
I20260811 22:33:23.989632 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:33:23.989938 19278 external_mini_cluster.cc:1428] Setting key 6a7502557647b4ad036584a05cfe98d9
I20260811 22:33:23.992583 11159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.211.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.189@KRBTEST.COM" created.
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:24.009306 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:23.995395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45915 (local address 127.0.0.1:39596)
0811 22:33:23.995971 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:23.996028 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:23.997379 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:23.998127 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:23.998133 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:23.998615 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:23.999011 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:23.999033 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.000239 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.000246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.001041 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.001055 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.001747 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.001751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.001938 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.002547 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.002563 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.004106 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.006367 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.006378 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.006389 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.006610 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.006925 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.006930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.006932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.007042 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.007617 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.007624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.007767 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.007935 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.008293 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":145,"thread_start_us":153,"threads_started":1}
I20260811 22:33:24.009306 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:23.995395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.18.211.190:45915)
0811 22:33:23.995973 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:23.995990 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:23.996149 (+ 159us) server_negotiation.cc:407] Connection header received
0811 22:33:23.997707 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:23.997712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:23.997900 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:23.998071 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:23.999179 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.000110 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.001161 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.001590 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.001795 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.004268 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.004278 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.004295 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.004326 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.006268 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.006723 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.006729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.006731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.006834 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.007134 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.007138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.007140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.007506 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.007647 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.007997 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.008240 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":454,"spinlock_wait_cycles":50432,"thread_start_us":163,"threads_started":1}
I20260811 22:33:24.011979 11159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:33:24.012701 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.012451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52778)
0811 22:33:24.012566 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.012636 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:33:24.012962 11062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
I20260811 22:33:24.013877 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.013694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52792)
0811 22:33:24.013785 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.013826 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:33:24.014060 11159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:43965: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab.
Entry for principal kudu/127.18.211.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab.
I20260811 22:33:24.018630 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:43965
--webserver_interface=127.18.211.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab
--principal=kudu/127.18.211.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45915,127.18.211.189:43965,127.18.211.188:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:24.063334 11159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } attempt: 1
I20260811 22:33:24.064270 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.063915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52808)
0811 22:33:24.064094 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.064169 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 22:33:24.065094 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.064861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52812)
0811 22:33:24.064974 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.065027 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:33:24.065256 11159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:43965: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
I20260811 22:33:24.133417 11159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } attempt: 2
I20260811 22:33:24.134379 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.134036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52820)
0811 22:33:24.134200 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.134277 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:33:24.134536 11170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:24.134912 11170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:24.135022 11170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:33:24.135257 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.135029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52828)
0811 22:33:24.135131 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.135188 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 22:33:24.135385 11159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:43965: Network error: Client connection negotiation failed: client connection to 127.18.211.189:43965: connect: Connection refused (error 111)
W20260811 22:33:24.139266 11170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:24.139324 11170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:24.139346 11170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:24.139366 11170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:24.139389 11170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:24.144521 11170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45915,127.18.211.189:43965,127.18.211.188:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab
--principal=kudu/127.18.211.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:43965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.211.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:24.145615 11170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:24.146616 11170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:24.155306 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:24.155306 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:24.155409 11186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:24.155519 11170 server_base.cc:1034] running on GCE node
I20260811 22:33:24.156217 11170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:24.157415 11170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:24.158576 11170 hybrid_clock.cc:648] HybridClock initialized: now 1786487604158548 us; error 42 us; skew 500 ppm
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:24.161384 11170 init.cc:377] Logged in from keytab as kudu/127.18.211.189@KRBTEST.COM (short username kudu)
I20260811 22:33:24.162287 11170 webserver.cc:460] Webserver started at http://127.18.211.189:32785/ using document root <none> and password file <none>
I20260811 22:33:24.162863 11170 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:24.162952 11170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:24.163170 11170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:24.165277 11170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data/instance:
uuid: "53b8de24cd19481d8ec343a77344b931"
format_stamp: "Formatted at 2026-08-11 22:33:24 on dist-test-slave-8hhm"
server_key: "74e8d14b11406b98a4cc6a792584db85"
server_key_iv: "19308311867f1ffba30925d8cc335ccd"
server_key_version: "encryptionkey@0"
I20260811 22:33:24.165872 11170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/wal/instance:
uuid: "53b8de24cd19481d8ec343a77344b931"
format_stamp: "Formatted at 2026-08-11 22:33:24 on dist-test-slave-8hhm"
server_key: "74e8d14b11406b98a4cc6a792584db85"
server_key_iv: "19308311867f1ffba30925d8cc335ccd"
server_key_version: "encryptionkey@0"
I20260811 22:33:24.169559 11170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:33:24.172014 11195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:24.173301 11170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:33:24.173432 11170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/wal
uuid: "53b8de24cd19481d8ec343a77344b931"
format_stamp: "Formatted at 2026-08-11 22:33:24 on dist-test-slave-8hhm"
server_key: "74e8d14b11406b98a4cc6a792584db85"
server_key_iv: "19308311867f1ffba30925d8cc335ccd"
server_key_version: "encryptionkey@0"
I20260811 22:33:24.173542 11170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:24.192430 11170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:24.195361 11170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:24.195573 11170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:24.202612 11170 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.189:43965
I20260811 22:33:24.202925 11274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.189:43965 every 8 connection(s)
I20260811 22:33:24.203728 11170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
I20260811 22:33:24.205997 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 11170
I20260811 22:33:24.206146 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-1/wal/instance
I20260811 22:33:24.206470 19278 external_mini_cluster.cc:1428] Setting key 5ec2fb613b6a41b28ee640530faef1af
I20260811 22:33:24.206815 11275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:24.211051 11275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.211.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.188@KRBTEST.COM" created.
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:24.225124 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.213066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.18.211.190:45915)
0811 22:33:24.213207 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.213245 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.214277 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 22:33:24.215201 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.215205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.215265 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.215364 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.216594 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.217113 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.218510 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.218806 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.218922 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.221433 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.221437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.221439 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.221464 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.222943 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.223525 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.223528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.223529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.223614 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.223970 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.223974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.223976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.224163 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.224272 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.224808 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.224945 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:33:24.225947 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.213611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45915 (local address 127.0.0.1:39602)
0811 22:33:24.214126 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.214169 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.214986 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.215536 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.215544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.215919 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.216404 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.216417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.217314 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.217318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.218382 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.218392 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.218997 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.219000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.219156 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.219798 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.219814 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.221255 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.223074 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.223080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.223095 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.223407 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.223707 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.223713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.223717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.223865 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.224273 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.224281 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.224537 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.224724 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.225059 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":151,"threads_started":1}
I20260811 22:33:24.226539 11159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } attempt: 3
I20260811 22:33:24.227537 11275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab.
Entry for principal kudu/127.18.211.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab.
I20260811 22:33:24.237071 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:42553
--webserver_interface=127.18.211.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab
--principal=kudu/127.18.211.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45915,127.18.211.189:43965,127.18.211.188:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:24.237459 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.227005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52844)
0811 22:33:24.227166 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.227180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.227251 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.227983 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.227987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.228176 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.228302 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.228311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.229566 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.229570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.230174 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.230184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.230907 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.230911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.231022 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.231550 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.231561 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.233085 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.235208 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.235214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.235218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.235535 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.235926 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.235933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.235938 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.236067 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.236650 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.236654 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.236778 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.237012 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.237172 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 22:33:24.238138 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.227180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.18.211.189:43965)
0811 22:33:24.227540 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.227549 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.227567 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:33:24.227602 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.227606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.227791 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.227930 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.228393 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.229448 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.230295 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.230783 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.230895 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.233269 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.233281 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.233297 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.233330 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.235089 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.235677 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.235685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.235687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.235787 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.236179 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.236192 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.236193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.236514 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.236661 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.237850 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.237996 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":258,"thread_start_us":174,"threads_started":1}
I20260811 22:33:24.238850 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.228054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52856)
0811 22:33:24.228176 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.228198 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.228266 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.228615 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.228618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.228788 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.228888 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.228899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.229871 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.229874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.230352 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.230359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.230899 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.230912 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.231046 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.231631 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.231650 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.233671 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.237037 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.237040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.237042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.237333 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.237654 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.237657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.237659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.237754 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.238260 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.238265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.238375 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.238498 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.238627 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":19}
I20260811 22:33:24.239040 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.228078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.18.211.189:43965)
0811 22:33:24.228332 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.228338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.228350 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:33:24.228419 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.228422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.228461 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.228606 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.228989 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.229609 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.230488 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.230786 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.230900 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.233905 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.233908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.233910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.233932 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.236899 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.237446 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.237449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.237451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.237539 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.237863 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.237870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.237874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.238114 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.238346 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.238570 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.238702 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
I20260811 22:33:24.240085 11159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:33:24.240252 11275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:33:24.240886 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.240609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40436)
0811 22:33:24.240743 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.240800 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 22:33:24.240923 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.240680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40438)
0811 22:33:24.240792 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.240870 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:33:24.241192 11198 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
I20260811 22:33:24.241832 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.241484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40446)
0811 22:33:24.241691 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.241752 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260811 22:33:24.242072 11159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:42553: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
I20260811 22:33:24.242136 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.241905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40450)
0811 22:33:24.242010 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.242067 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:33:24.242269 11275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:42553: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
I20260811 22:33:24.273267 11159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } attempt: 1
I20260811 22:33:24.274211 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.273893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40458)
0811 22:33:24.274052 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.274115 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:33:24.274948 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.274748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40460)
0811 22:33:24.274852 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.274896 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:33:24.275161 11159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:42553: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
I20260811 22:33:24.291538 11275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } attempt: 1
I20260811 22:33:24.292398 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.292105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40472)
0811 22:33:24.292251 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.292319 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:33:24.293125 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.292920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40476)
0811 22:33:24.293022 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.293070 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:33:24.293273 11275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:42553: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
I20260811 22:33:24.350333 11159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } attempt: 2
I20260811 22:33:24.351294 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.350930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40484)
0811 22:33:24.351144 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.351210 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 22:33:24.352317 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.352117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40498)
0811 22:33:24.352218 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.352264 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:33:24.352455 11159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:42553: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
W20260811 22:33:24.356093 11291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:24.356364 11291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:24.356405 11291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:24.360720 11291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:24.360786 11291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:24.360807 11291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:24.360824 11291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:24.360839 11291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:24.361423 11275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } attempt: 2
I20260811 22:33:24.362179 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.361900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40512)
0811 22:33:24.362033 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.362104 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:33:24.362912 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.362707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40524)
0811 22:33:24.362807 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.362854 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 22:33:24.363099 11275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:42553: Network error: Client connection negotiation failed: client connection to 127.18.211.188:42553: connect: Connection refused (error 111)
I20260811 22:33:24.366798 11291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45915,127.18.211.189:43965,127.18.211.188:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab
--principal=kudu/127.18.211.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.211.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:24.367973 11291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:24.369027 11291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:24.377620 11298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:24.377647 11297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:24.377620 11307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:24.377830 11291 server_base.cc:1034] running on GCE node
I20260811 22:33:24.378540 11291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:24.379683 11291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:24.380853 11291 hybrid_clock.cc:648] HybridClock initialized: now 1786487604380817 us; error 48 us; skew 500 ppm
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:24.383685 11291 init.cc:377] Logged in from keytab as kudu/127.18.211.188@KRBTEST.COM (short username kudu)
I20260811 22:33:24.384546 11291 webserver.cc:460] Webserver started at http://127.18.211.188:37803/ using document root <none> and password file <none>
I20260811 22:33:24.385098 11291 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:24.385190 11291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:24.385433 11291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:24.387458 11291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data/instance:
uuid: "c727e5b224f34234b65c2cfa04e7579e"
format_stamp: "Formatted at 2026-08-11 22:33:24 on dist-test-slave-8hhm"
server_key: "a6e5ca7485e225a4ec49fa9e635d02c7"
server_key_iv: "ecf303f3afb0ee847bfc94d45d6f71f9"
server_key_version: "encryptionkey@0"
I20260811 22:33:24.387984 11291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/wal/instance:
uuid: "c727e5b224f34234b65c2cfa04e7579e"
format_stamp: "Formatted at 2026-08-11 22:33:24 on dist-test-slave-8hhm"
server_key: "a6e5ca7485e225a4ec49fa9e635d02c7"
server_key_iv: "ecf303f3afb0ee847bfc94d45d6f71f9"
server_key_version: "encryptionkey@0"
I20260811 22:33:24.391743 11291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:33:24.394158 11324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:24.395437 11291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:33:24.395570 11291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/wal
uuid: "c727e5b224f34234b65c2cfa04e7579e"
format_stamp: "Formatted at 2026-08-11 22:33:24 on dist-test-slave-8hhm"
server_key: "a6e5ca7485e225a4ec49fa9e635d02c7"
server_key_iv: "ecf303f3afb0ee847bfc94d45d6f71f9"
server_key_version: "encryptionkey@0"
I20260811 22:33:24.395682 11291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:24.410434 11291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:24.413283 11291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:24.413473 11291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:24.420579 11291 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.188:42553
I20260811 22:33:24.420881 11416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.188:42553 every 8 connection(s)
I20260811 22:33:24.421599 11291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
I20260811 22:33:24.424486 11417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:24.425889 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 11291
I20260811 22:33:24.425987 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487433619239-19278-0/minicluster-data/master-2/wal/instance
I20260811 22:33:24.426316 19278 external_mini_cluster.cc:1428] Setting key 8ccfe05eafc80f8ec663d0b4497728ed
I20260811 22:33:24.429064 11417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:24.440708 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.430988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.18.211.190:45915)
0811 22:33:24.431136 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.431141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.431449 (+ 308us) server_negotiation.cc:407] Connection header received
0811 22:33:24.431577 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.431580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.431629 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.431713 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.432467 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.433112 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.434195 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.434455 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.434568 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.436910 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.436913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.436916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.436946 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.438780 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.439292 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.439294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.439295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.439370 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.439631 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.439633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.439634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.439863 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.439965 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.440406 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.440521 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:33:24.441504 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.430927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45915 (local address 127.0.0.1:39610)
0811 22:33:24.431316 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.431363 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.431482 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.431794 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.431799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.432223 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.432340 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.432346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.433259 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.433262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.434078 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.434086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.434608 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.434613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.434716 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.435271 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.435294 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.436769 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.438872 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.438881 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.438888 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.439173 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.439450 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.439453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.439455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.439538 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.439979 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.439986 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.440122 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.440853 (+ 731us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.441024 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"spinlock_wait_cycles":3712,"thread_start_us":131,"threads_started":1}
I20260811 22:33:24.441509 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.427666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.18.211.188:42553)
0811 22:33:24.428117 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.428136 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.428180 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:33:24.429034 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.429058 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.429505 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.429889 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.430548 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.431925 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.432964 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.433591 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.433760 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.436325 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.436331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.436341 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.436363 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.438201 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.438696 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.438701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.438706 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.438805 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.439108 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.439111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.439112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.439664 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.439803 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.440126 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.440461 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"spinlock_wait_cycles":14080,"thread_start_us":108,"threads_started":1}
I20260811 22:33:24.442911 11417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:24.451625 11159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } attempt: 3
I20260811 22:33:24.452705 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.443422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:43965 (local address 127.0.0.1:52860)
0811 22:33:24.443546 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.443562 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.443632 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.444155 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.444158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.444337 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.444510 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.444519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.445506 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.445509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.446321 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.446332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.446848 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.446852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.446976 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.447407 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.447434 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.448825 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.450780 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.450783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.450785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.451025 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.451298 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.451302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.451304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.451401 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.452119 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.452124 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.452233 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.452384 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.452521 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 22:33:24.452775 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.443508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.18.211.189:43965)
0811 22:33:24.443891 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.443895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.443907 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:33:24.443967 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.443969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.444020 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.444100 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.444642 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.445404 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.446444 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.446736 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.446830 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.448967 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.448970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.448972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.448994 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.450672 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.451119 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.451122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.451123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.451207 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.451496 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.451499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.451501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.451789 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.451899 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.452462 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.452586 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303}
I20260811 22:33:24.453537 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.443883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.18.211.190:45915)
0811 22:33:24.444002 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.444007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.444018 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:33:24.444050 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.444055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.444098 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.444164 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.444888 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.445599 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.446602 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.446902 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.446982 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.449673 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.449676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.449678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.449706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.451364 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.451893 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.451898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.451900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.451993 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.452332 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.452339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.452343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.452579 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.453169 (+ 590us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.453270 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.453406 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:33:24.454313 11417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:33:24.454326 11275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } attempt: 3
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:33:24 dist-test-slave-8hhm krb5kdc[11010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
I20260811 22:33:24.462078 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.452186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.18.211.188:42553)
0811 22:33:24.452307 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.452312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.452325 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:33:24.452461 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.452464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.452508 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.453298 (+ 790us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.453416 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.454583 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.455668 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.456013 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.456523 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.458402 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.458406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.458408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.458433 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.460172 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.460740 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.460743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.460745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.460832 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.461169 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.461172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.461174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.461435 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.461554 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.461823 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.461933 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":46}
I20260811 22:33:24.462078 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.452098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40542)
0811 22:33:24.452270 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.452285 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.452356 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.452624 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.452627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.452838 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.452954 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.452961 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.454716 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.454720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.455517 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.455531 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.456156 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.456162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.456269 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.456721 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.456735 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.458237 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.460307 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.460311 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.460313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.460600 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.460956 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.460959 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.460961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.461048 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.461547 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.461550 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.461652 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.461769 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.461909 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 22:33:24.465662 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.455023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52868 (local address 127.18.211.189:43965)
0811 22:33:24.455127 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.455130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.455142 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:33:24.455269 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.455272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.455314 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.455361 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.456253 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.456989 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.458079 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.458397 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.458472 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.461858 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.461862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.461864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.461890 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.463812 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.464313 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.464318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.464320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.464407 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.464681 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.464685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.464687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.464920 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.465142 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.465348 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.465476 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:33:24.466403 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.455268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40560)
0811 22:33:24.455362 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.455375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.455436 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.456213 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.456219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.456385 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.456490 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.456497 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.457554 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.457557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.458297 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.458306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.459581 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.459586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.459683 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.460154 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.460174 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.462266 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.464841 (+ 2575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.464844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.464846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.465094 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.465436 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.465442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.465444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.465524 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.465920 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.465922 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.466012 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.466124 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.466248 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":3}
I20260811 22:33:24.466979 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.455426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.18.211.188:42553)
0811 22:33:24.455983 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.455996 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.456007 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:33:24.456037 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.456039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.456073 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.456586 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.456678 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.457447 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.458481 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.459442 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.459896 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.462386 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.462393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.462396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.462422 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.464726 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.465203 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.465206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.465208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.465336 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.465620 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.465622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.465624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.465814 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.465928 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.466678 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.466807 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":533,"server-negotiator.queue_time_us":488,"thread_start_us":92,"threads_started":1}
I20260811 22:33:24.467157 11159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: Bootstrap starting.
I20260811 22:33:24.469157 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:33:24.469941 11159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:24.469913 11417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: Bootstrap starting.
I20260811 22:33:24.470820 11159 log.cc:824] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:24.472754 11417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:24.473380 11159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: No bootstrap required, opened a new log
I20260811 22:33:24.473649 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.454655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:42553 (local address 127.0.0.1:40544)
0811 22:33:24.454791 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:24.454805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:24.454882 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:24.455790 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:24.455793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:24.455969 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:24.456070 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.456079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.457437 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.457440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.458261 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:24.458274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.460159 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.460163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:24.460268 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.460749 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:24.460763 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:24.462709 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:24.471914 (+ 9205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.471918 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.471920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.472170 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.472533 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:24.472536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:24.472538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:24.472632 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:24.473125 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:24.473128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:24.473248 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 22:33:24.473353 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:33:24.473504 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:33:24.473687 11417 log.cc:824] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:24.474254 11424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.454904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.18.211.188:42553)
0811 22:33:24.455560 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.455573 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.455585 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:33:24.455613 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.455616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.455656 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.455759 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.456195 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.457300 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.458388 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.459632 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.459782 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.462870 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.462895 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.462900 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.462929 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.471779 (+ 8850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.472291 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.472294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.472296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.472415 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.472756 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.472759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.472761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.472982 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.473078 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.473968 (+ 890us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.474085 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":282,"server-negotiator.queue_time_us":375,"thread_start_us":132,"threads_started":1}
I20260811 22:33:24.476557 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931: Bootstrap starting.
I20260811 22:33:24.476830 11159 raft_consensus.cc:348] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.477021 11159 raft_consensus.cc:374] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:24.477092 11159 raft_consensus.cc:729] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acbb7875964d4e9bb61b6ccf5c0c3273, State: Initialized, Role: FOLLOWER
I20260811 22:33:24.477670 11159 consensus_queue.cc:260] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.478597 11417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: No bootstrap required, opened a new log
W20260811 22:33:24.478957 11076 tablet.cc:2369] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:33:24.479306 11275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:24.479795 11159 sys_catalog.cc:564] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:24.480197 11275 log.cc:824] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:24.483654 11417 raft_consensus.cc:348] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.483933 11417 raft_consensus.cc:374] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:24.484001 11417 raft_consensus.cc:729] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c727e5b224f34234b65c2cfa04e7579e, State: Initialized, Role: FOLLOWER
I20260811 22:33:24.484503 11417 consensus_queue.cc:260] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.486022 11417 sys_catalog.cc:564] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:24.484910 11032 master_runner.cc:186] Error getting master registration for 127.18.211.189:43965: OK, instance_id { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" instance_seqno: 1786487604200608 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:33:24.489142 11032 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:33:24.491533 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.472922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.18.211.188:42553)
0811 22:33:24.473272 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.473275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.479567 (+ 6292us) server_negotiation.cc:407] Connection header received
0811 22:33:24.480119 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.480123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.480179 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.480230 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.482184 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.483069 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.484250 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.484600 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.484684 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.486621 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.486625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.486628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.486651 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.488714 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.490027 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.490033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.490035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.490139 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.490423 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.490426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.490428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.490680 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.490845 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.491226 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.491371 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260811 22:33:24.492384 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.472697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39632 (local address 127.18.211.190:45915)
0811 22:33:24.473149 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.473154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.477554 (+ 4400us) server_negotiation.cc:407] Connection header received
0811 22:33:24.477834 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.477837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.477886 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.477941 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.482208 (+ 4267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.482915 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.485401 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.485738 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.485817 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.488665 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.488669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.488672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.488711 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.490350 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.491002 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.491006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.491008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.491099 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.491407 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.491411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.491413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.491643 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.491784 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.492127 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.492248 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
W20260811 22:33:24.493464 11470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:33:24.493543 11470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:33:24.493742 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.493860 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:33:24.495517 11472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.496035 11472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:33:24.496753 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931: No bootstrap required, opened a new log
I20260811 22:33:24.493664 11291 master_runner.cc:186] Error getting master registration for 127.18.211.189:43965: OK, instance_id { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" instance_seqno: 1786487604200608 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:33:24.496984 11291 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:33:24.499151 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:24.472819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.18.211.189:43965)
0811 22:33:24.473055 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:24.473060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:24.478926 (+ 5866us) server_negotiation.cc:407] Connection header received
0811 22:33:24.479844 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:24.479847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:24.479899 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:24.479950 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:24.484112 (+ 4162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.484785 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.487116 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:24.487484 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:24.487559 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:24.495011 (+ 7452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:24.495017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:24.495020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:24.495049 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:24.496802 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.497360 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.497364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.497366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.497460 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:24.497800 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:24.497804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:24.497805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:24.498040 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:24.498168 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:24.498842 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 22:33:24.498991 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 22:33:24.500874 11275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.501094 11275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:24.501151 11275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b8de24cd19481d8ec343a77344b931, State: Initialized, Role: FOLLOWER
I20260811 22:33:24.501644 11275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.503090 11275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:33:24.503394 11496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:33:24.503479 11496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 22:33:24.510008 11518 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:33:24.510102 11518 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:33:24.513365 11501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.513729 11501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:33:24.515522 11170 master_runner.cc:263] No leader master found from master 53b8de24cd19481d8ec343a77344b931
I20260811 22:33:24.515601 11170 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:33:24.664937 11337 tablet.cc:2369] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:33:24.697146 11205 tablet.cc:2369] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:33:24.819701 11501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:24.819911 11501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.820820 11501 leader_election.cc:288] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acbb7875964d4e9bb61b6ccf5c0c3273 (127.18.211.190:45915), c727e5b224f34234b65c2cfa04e7579e (127.18.211.188:42553)
I20260811 22:33:24.821417 11374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "53b8de24cd19481d8ec343a77344b931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c727e5b224f34234b65c2cfa04e7579e" is_pre_election: true
I20260811 22:33:24.821451 11120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "53b8de24cd19481d8ec343a77344b931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" is_pre_election: true
I20260811 22:33:24.821760 11120 raft_consensus.cc:2454] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53b8de24cd19481d8ec343a77344b931 in term 0.
I20260811 22:33:24.821766 11374 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53b8de24cd19481d8ec343a77344b931 in term 0.
I20260811 22:33:24.822242 11201 leader_election.cc:302] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53b8de24cd19481d8ec343a77344b931, acbb7875964d4e9bb61b6ccf5c0c3273, c727e5b224f34234b65c2cfa04e7579e; no voters:
I20260811 22:33:24.822522 11501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:33:24.822613 11501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:33:24.822659 11501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:24.823748 11501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.824101 11501 leader_election.cc:288] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [CANDIDATE]: Term 1 election: Requested vote from peers acbb7875964d4e9bb61b6ccf5c0c3273 (127.18.211.190:45915), c727e5b224f34234b65c2cfa04e7579e (127.18.211.188:42553)
I20260811 22:33:24.824443 11120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "53b8de24cd19481d8ec343a77344b931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273"
I20260811 22:33:24.824456 11374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "53b8de24cd19481d8ec343a77344b931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c727e5b224f34234b65c2cfa04e7579e"
I20260811 22:33:24.824584 11374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:24.824591 11120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:24.825695 11374 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53b8de24cd19481d8ec343a77344b931 in term 1.
I20260811 22:33:24.825726 11120 raft_consensus.cc:2454] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53b8de24cd19481d8ec343a77344b931 in term 1.
I20260811 22:33:24.826061 11198 leader_election.cc:302] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53b8de24cd19481d8ec343a77344b931, acbb7875964d4e9bb61b6ccf5c0c3273, c727e5b224f34234b65c2cfa04e7579e; no voters:
I20260811 22:33:24.826274 11501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:24.826470 11501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [term 1 LEADER]: Becoming Leader. State: Replica: 53b8de24cd19481d8ec343a77344b931, State: Running, Role: LEADER
I20260811 22:33:24.826798 11501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } }
I20260811 22:33:24.828656 11522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b8de24cd19481d8ec343a77344b931. Latest consensus state: current_term: 1 leader_uuid: "53b8de24cd19481d8ec343a77344b931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.828789 11522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:24.829116 11524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:24.830018 11524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:24.834513 11374 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [term 1 FOLLOWER]: Refusing update from remote peer 53b8de24cd19481d8ec343a77344b931: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:24.834513 11120 raft_consensus.cc:1261] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [term 1 FOLLOWER]: Refusing update from remote peer 53b8de24cd19481d8ec343a77344b931: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:24.835176 11501 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:24.835372 11522 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:24.839465 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b8de24cd19481d8ec343a77344b931. Latest consensus state: current_term: 1 leader_uuid: "53b8de24cd19481d8ec343a77344b931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.839593 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:33:24.839569 11472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b8de24cd19481d8ec343a77344b931. Latest consensus state: current_term: 1 leader_uuid: "53b8de24cd19481d8ec343a77344b931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.839691 11472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:33:24.842726 11534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53b8de24cd19481d8ec343a77344b931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.842926 11534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:24.843123 11472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53b8de24cd19481d8ec343a77344b931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.843116 11501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53b8de24cd19481d8ec343a77344b931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.843215 11472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:33:24.843222 11501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:24.843392 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53b8de24cd19481d8ec343a77344b931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb7875964d4e9bb61b6ccf5c0c3273" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45915 } } peers { permanent_uuid: "53b8de24cd19481d8ec343a77344b931" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 43965 } } peers { permanent_uuid: "c727e5b224f34234b65c2cfa04e7579e" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 42553 } } }
I20260811 22:33:24.843499 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:33:24.843508 11528 mvcc.cc:205] Tried to move back new op lower bound from 7317453229395443712 to 7317453229373026304. Current Snapshot: MvccSnapshot[applied={T|T < 7317453229395443712}]
I20260811 22:33:24.843820 11530 mvcc.cc:205] Tried to move back new op lower bound from 7317453229395443712 to 7317453229373026304. Current Snapshot: MvccSnapshot[applied={T|T < 7317453229395443712}]
I20260811 22:33:24.844293 11524 catalog_manager.cc:1347] Generated new cluster ID: 2ea916e0be204b2ba5795f8c17bb5d24
I20260811 22:33:24.844353 11524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:33:24.857904 11524 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:33:24.858883 11524 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:33:24.867569 11524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b8de24cd19481d8ec343a77344b931: Generated new TSK 0
I20260811 22:33:24.868641 11524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:33:25.495056 11470 catalog_manager.cc:1259] Loaded cluster ID: 2ea916e0be204b2ba5795f8c17bb5d24
I20260811 22:33:25.495153 11470 catalog_manager.cc:1552] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: loading cluster ID for follower catalog manager: success
I20260811 22:33:25.496908 11470 catalog_manager.cc:1574] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: acquiring CA information for follower catalog manager: success
I20260811 22:33:25.498180 11470 catalog_manager.cc:1602] T 00000000000000000000000000000000 P acbb7875964d4e9bb61b6ccf5c0c3273: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:33:25.504776 11496 catalog_manager.cc:1259] Loaded cluster ID: 2ea916e0be204b2ba5795f8c17bb5d24
I20260811 22:33:25.504841 11496 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: loading cluster ID for follower catalog manager: success
I20260811 22:33:25.506528 11496 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: acquiring CA information for follower catalog manager: success
I20260811 22:33:25.507666 11496 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c727e5b224f34234b65c2cfa04e7579e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:33:25.947716 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 11032
I20260811 22:33:25.969409 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 11170
I20260811 22:33:25.990064 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 11291
2026-08-11T22:33:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3647 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:33:26 dist-test-slave-8hhm krb5kdc[11549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:33:26 dist-test-slave-8hhm krb5kdc[11549](info): set up 2 sockets
Aug 11 22:33:26 dist-test-slave-8hhm krb5kdc[11549](info): commencing operation
krb5kdc: starting...
W20260811 22:33:27.293836 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:33:27 dist-test-slave-8hhm krb5kdc[11549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:33:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:33:27.424433 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44617
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:44617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:27.537801 11568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:27.538089 11568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:27.538132 11568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:27.542165 11568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:27.542220 11568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:27.542239 11568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:27.542290 11568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:27.542323 11568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:27.547674 11568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:44617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:27.548733 11568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:27.549820 11568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:27.558401 11580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:27.558362 11575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:27.558429 11576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:27.558761 11568 server_base.cc:1034] running on GCE node
I20260811 22:33:27.559347 11568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:27.560418 11568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:27.561610 11568 hybrid_clock.cc:648] HybridClock initialized: now 1786487607561578 us; error 53 us; skew 500 ppm
Aug 11 22:33:27 dist-test-slave-8hhm krb5kdc[11549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487607, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:27.564407 11568 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:27.565378 11568 webserver.cc:460] Webserver started at http://127.18.211.190:34553/ using document root <none> and password file <none>
I20260811 22:33:27.565941 11568 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:27.566035 11568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:27.566277 11568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:27.568344 11568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "65d56ea0d4354035b01b27d709184f63"
format_stamp: "Formatted at 2026-08-11 22:33:27 on dist-test-slave-8hhm"
server_key: "6bf8e8726df81e3cc2d3f8585565748c"
server_key_iv: "8b50cb5fa23578d2e8569385931d6ef1"
server_key_version: "encryptionkey@0"
I20260811 22:33:27.568919 11568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "65d56ea0d4354035b01b27d709184f63"
format_stamp: "Formatted at 2026-08-11 22:33:27 on dist-test-slave-8hhm"
server_key: "6bf8e8726df81e3cc2d3f8585565748c"
server_key_iv: "8b50cb5fa23578d2e8569385931d6ef1"
server_key_version: "encryptionkey@0"
I20260811 22:33:27.572623 11568 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 22:33:27.575070 11598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:27.576287 11568 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:33:27.576436 11568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "65d56ea0d4354035b01b27d709184f63"
format_stamp: "Formatted at 2026-08-11 22:33:27 on dist-test-slave-8hhm"
server_key: "6bf8e8726df81e3cc2d3f8585565748c"
server_key_iv: "8b50cb5fa23578d2e8569385931d6ef1"
server_key_version: "encryptionkey@0"
I20260811 22:33:27.576555 11568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:27.591224 11568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:27.594779 11568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:27.595005 11568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:27.602435 11568 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:44617
I20260811 22:33:27.602461 11676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:44617 every 8 connection(s)
I20260811 22:33:27.603447 11568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:27.606235 11677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:27.611650 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 11568
I20260811 22:33:27.611797 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:33:27.612097 19278 external_mini_cluster.cc:1428] Setting key 41d2c25847d23416e8f9d2727f4f5ea6
I20260811 22:33:27.612083 11677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63: Bootstrap starting.
I20260811 22:33:27.614604 11677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:27.615348 11677 log.cc:824] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:27.617200 11677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63: No bootstrap required, opened a new log
Aug 11 22:33:27 dist-test-slave-8hhm krb5kdc[11549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:27.620813 11677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d56ea0d4354035b01b27d709184f63" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44617 } }
I20260811 22:33:27.621100 11677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:27.621171 11677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d56ea0d4354035b01b27d709184f63, State: Initialized, Role: FOLLOWER
I20260811 22:33:27.621672 11677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d56ea0d4354035b01b27d709184f63" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44617 } }
I20260811 22:33:27.621805 11677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:33:27.621929 11677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:33:27.622052 11677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:27.622872 11677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d56ea0d4354035b01b27d709184f63" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44617 } }
I20260811 22:33:27.623217 11677 leader_election.cc:302] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d56ea0d4354035b01b27d709184f63; no voters:
I20260811 22:33:27.623517 11677 leader_election.cc:288] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:33:27.623637 11687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:27.623903 11687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [term 1 LEADER]: Becoming Leader. State: Replica: 65d56ea0d4354035b01b27d709184f63, State: Running, Role: LEADER
I20260811 22:33:27.624270 11687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d56ea0d4354035b01b27d709184f63" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44617 } }
I20260811 22:33:27.624856 11677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:27.626732 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:27.613447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.18.211.190:44617)
0811 22:33:27.613870 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:27.613886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:27.613920 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:33:27.614537 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:27.614555 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:27.614928 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:27.615277 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:27.616094 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:27.617047 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:27.618038 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:27.618564 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:27.618740 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:27.621553 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:27.621559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:27.621581 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:27.621607 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:27.623467 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:27.624027 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:27.624033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:27.624039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:27.624171 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:27.624529 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:27.624532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:27.624534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:27.625049 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:27.625310 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:27.625598 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 22:33:27.625886 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":142,"threads_started":1}
I20260811 22:33:27.628497 11688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d56ea0d4354035b01b27d709184f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d56ea0d4354035b01b27d709184f63" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44617 } } }
I20260811 22:33:27.628628 11688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:27.628896 11690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d56ea0d4354035b01b27d709184f63. Latest consensus state: current_term: 1 leader_uuid: "65d56ea0d4354035b01b27d709184f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d56ea0d4354035b01b27d709184f63" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44617 } } }
I20260811 22:33:27.629019 11690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:27.630185 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 22:33:27.630836 11712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:33:27.630923 11712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:33:27.631036 11715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:27.631825 11715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:27.636979 11715 catalog_manager.cc:1347] Generated new cluster ID: cf62ad3971d14accb0979f05e3f95248
I20260811 22:33:27.637044 11715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:33:27.644075 11715 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:33:27.645288 11715 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:33:27.656508 11715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d56ea0d4354035b01b27d709184f63: Generated new TSK 0
I20260811 22:33:27.657287 11715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:33:27.686169 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 11568
2026-08-11T22:33:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1697 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:33:27 dist-test-slave-8hhm krb5kdc[11722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:33:27 dist-test-slave-8hhm krb5kdc[11722](info): set up 2 sockets
Aug 11 22:33:27 dist-test-slave-8hhm krb5kdc[11722](info): commencing operation
krb5kdc: starting...
W20260811 22:33:28.998670 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:33:29 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:33:29.128649 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44989
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:44989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:29.242524 11745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:29.242812 11745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:29.242880 11745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:29.246896 11745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:29.246975 11745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:29.247008 11745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:29.247104 11745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:29.247165 11745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:29.252305 11745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:44989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:29.253360 11745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:29.254393 11745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:29.263247 11752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:29.263257 11751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:29.263289 11745 server_base.cc:1034] running on GCE node
W20260811 22:33:29.263327 11755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:29.264201 11745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:29.265432 11745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:29.266600 11745 hybrid_clock.cc:648] HybridClock initialized: now 1786487609266580 us; error 31 us; skew 500 ppm
Aug 11 22:33:29 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487609, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:29.269511 11745 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:29.270406 11745 webserver.cc:460] Webserver started at http://127.18.211.190:39345/ using document root <none> and password file <none>
I20260811 22:33:29.270983 11745 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:29.271075 11745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:29.271333 11745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:29.273530 11745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "d02403a59d954c6bb82894ce851b2155"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "7ebedd3b2db3cdfc5da96369ce8da45f"
server_key_iv: "c40a164f44375289870e06f822b42d7d"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.274114 11745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "d02403a59d954c6bb82894ce851b2155"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "7ebedd3b2db3cdfc5da96369ce8da45f"
server_key_iv: "c40a164f44375289870e06f822b42d7d"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.278002 11745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:33:29.280465 11770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:29.281809 11745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:33:29.281955 11745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d02403a59d954c6bb82894ce851b2155"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "7ebedd3b2db3cdfc5da96369ce8da45f"
server_key_iv: "c40a164f44375289870e06f822b42d7d"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.282069 11745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:29.303951 11745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:29.307467 11745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:29.307710 11745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:29.314818 11745 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:44989
I20260811 22:33:29.315131 11845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:44989 every 8 connection(s)
I20260811 22:33:29.315902 11745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:29.318773 11846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:29.324244 11846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Bootstrap starting.
I20260811 22:33:29.325728 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 11745
I20260811 22:33:29.325860 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:33:29.326164 19278 external_mini_cluster.cc:1428] Setting key 5494f7110799e7d677834943e4a78e75
I20260811 22:33:29.326988 11846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:29.327848 11846 log.cc:824] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:29.330035 11846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: No bootstrap required, opened a new log
I20260811 22:33:29.334244 11846 raft_consensus.cc:348] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:29.334453 11846 raft_consensus.cc:374] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:29.334486 11846 raft_consensus.cc:729] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02403a59d954c6bb82894ce851b2155, State: Initialized, Role: FOLLOWER
I20260811 22:33:29.334939 11846 consensus_queue.cc:260] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:29.335084 11846 raft_consensus.cc:388] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:33:29.335156 11846 raft_consensus.cc:482] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:33:29.335268 11846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 22:33:29 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:29.336441 11846 raft_consensus.cc:504] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:29.336784 11846 leader_election.cc:302] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d02403a59d954c6bb82894ce851b2155; no voters:
I20260811 22:33:29.337131 11846 leader_election.cc:288] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:33:29.337203 11853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:29.337446 11853 raft_consensus.cc:686] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 LEADER]: Becoming Leader. State: Replica: d02403a59d954c6bb82894ce851b2155, State: Running, Role: LEADER
I20260811 22:33:29.337801 11853 consensus_queue.cc:237] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:29.338450 11846 sys_catalog.cc:564] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:29.339512 11854 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d02403a59d954c6bb82894ce851b2155. Latest consensus state: current_term: 1 leader_uuid: "d02403a59d954c6bb82894ce851b2155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } } }
I20260811 22:33:29.339660 11854 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:29.340087 11873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:29.340708 11855 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d02403a59d954c6bb82894ce851b2155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } } }
I20260811 22:33:29.340927 11855 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:29.342574 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:29.327294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.18.211.190:44989)
0811 22:33:29.327805 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:29.327825 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:29.327869 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:33:29.328695 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:29.328719 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:29.329101 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:29.329503 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:29.330172 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.331632 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.332480 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.333334 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.333497 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:29.336411 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:29.336416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:29.336429 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:29.336456 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:29.338484 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:29.339004 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:29.339011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:29.339018 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:29.339177 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:29.339567 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:29.339570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:29.339572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:29.340104 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:29.340322 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:29.341248 (+ 926us) server_negotiation.cc:299] Negotiation successful
0811 22:33:29.341590 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":169,"threads_started":1}
I20260811 22:33:29.343559 11873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:29.349049 11873 catalog_manager.cc:1347] Generated new cluster ID: a6edb518b12d45f5ba3aa3dee6103bae
I20260811 22:33:29.349129 11873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:33:29.368948 11873 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:33:29.369925 11873 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:33:29.385268 11873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Generated new TSK 0
I20260811 22:33:29.386060 11873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:33:29.418529 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44989
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:29.532764 11888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:29.533028 11888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:29.533070 11888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:29.537179 11888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:29.537259 11888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:29.537315 11888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:33:29.542670 11888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:29.543681 11888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:29.544715 11888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:29.554589 11898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:29.554844 11888 server_base.cc:1034] running on GCE node
W20260811 22:33:29.554672 11899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:29.554549 11904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:29.555428 11888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:29.556093 11888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:29.557300 11888 hybrid_clock.cc:648] HybridClock initialized: now 1786487609557277 us; error 29 us; skew 500 ppm
Aug 11 22:33:29 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487609, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:29.560199 11888 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:33:29.561019 11888 webserver.cc:460] Webserver started at http://127.18.211.129:34943/ using document root <none> and password file <none>
I20260811 22:33:29.561653 11888 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:29.561751 11888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:29.562003 11888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:29.564023 11888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "83205a60f3144ed3b072429678f08ff5"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "196a9d0706b88c1a66c9bf977329268f"
server_key_iv: "d5acff0820ee6b9fa326a20d67368d34"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.564585 11888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "83205a60f3144ed3b072429678f08ff5"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "196a9d0706b88c1a66c9bf977329268f"
server_key_iv: "d5acff0820ee6b9fa326a20d67368d34"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.568593 11888 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:33:29.571015 11919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:29.572160 11888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:33:29.572296 11888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "83205a60f3144ed3b072429678f08ff5"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "196a9d0706b88c1a66c9bf977329268f"
server_key_iv: "d5acff0820ee6b9fa326a20d67368d34"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.572405 11888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:29.610357 11888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:29.613821 11888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:29.614027 11888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:29.614638 11888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:29.615568 11888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:29.615638 11888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:29.615713 11888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:29.615757 11888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:29.624594 11888 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:44781
I20260811 22:33:29.624625 12113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:44781 every 8 connection(s)
I20260811 22:33:29.625530 11888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
Aug 11 22:33:29 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487609, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:29.635790 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 11888
I20260811 22:33:29.635936 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:33:29.636229 19278 external_mini_cluster.cc:1428] Setting key 3340b72d2c92a6304ce395bd59030ca5
I20260811 22:33:29.639312 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:29.627243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:45755 (local address 127.18.211.190:44989)
0811 22:33:29.627375 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:29.627379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:29.628171 (+ 792us) server_negotiation.cc:407] Connection header received
0811 22:33:29.629606 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:29.629609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:29.629668 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:29.629751 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:29.630875 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.631406 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.632558 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.632808 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.632915 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:29.635396 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:29.635400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:29.635402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:29.635428 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:29.637016 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:29.637595 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:29.637600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:29.637602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:29.637703 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:29.638098 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:29.638100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:29.638101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:29.638289 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:29.638368 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:29.638983 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 22:33:29.639186 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:33:29.640367 12117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:29.627601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:45755)
0811 22:33:29.628027 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:29.628069 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:29.629403 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:29.629921 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:29.629928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:29.630270 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:29.630728 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:29.630742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.631544 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.631547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:29.632419 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:29.632429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.632967 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.632971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:29.633131 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:29.633707 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:29.633723 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:29.635131 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:29.637130 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:29.637135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:29.637146 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:29.637488 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:29.637814 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:29.637817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:29.637819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:29.638005 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:29.638404 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:29.638413 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:29.638721 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 22:33:29.638930 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:33:29.639287 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
I20260811 22:33:29.641811 12114 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44989
I20260811 22:33:29.642045 12114 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:29.642505 12114 heartbeater.cc:511] Master 127.18.211.190:44989 requested a full tablet report, sending...
I20260811 22:33:29.643973 11794 ts_manager.cc:194] Registered new tserver with Master: 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781)
I20260811 22:33:29.645056 11794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:45755
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:33:29.666375 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44989
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:29.778749 12124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:29.778993 12124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:29.779033 12124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:29.783154 12124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:29.783219 12124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:29.783274 12124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:33:29.788420 12124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:29.789444 12124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:29.790393 12124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:29.800060 12138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:29.800076 12136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:29.800050 12135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:29.800454 12124 server_base.cc:1034] running on GCE node
I20260811 22:33:29.801172 12124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:29.801950 12124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:29.803184 12124 hybrid_clock.cc:648] HybridClock initialized: now 1786487609803160 us; error 33 us; skew 500 ppm
Aug 11 22:33:29 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487609, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:29.806216 12124 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:33:29.807114 12124 webserver.cc:460] Webserver started at http://127.18.211.130:33563/ using document root <none> and password file <none>
I20260811 22:33:29.807703 12124 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:29.807799 12124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:29.808045 12124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:29.810412 12124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "221d520f4bc144d680a536275a4239d7"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "b07e4031899dd6c9031ce6db7cfcf6c7"
server_key_iv: "4278be1ae00f5455a7f956ff5a7241a9"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.811002 12124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "221d520f4bc144d680a536275a4239d7"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "b07e4031899dd6c9031ce6db7cfcf6c7"
server_key_iv: "4278be1ae00f5455a7f956ff5a7241a9"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.814904 12124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:33:29.817435 12145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:29.818655 12124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:33:29.818802 12124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "221d520f4bc144d680a536275a4239d7"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "b07e4031899dd6c9031ce6db7cfcf6c7"
server_key_iv: "4278be1ae00f5455a7f956ff5a7241a9"
server_key_version: "encryptionkey@0"
I20260811 22:33:29.818920 12124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:29.836390 12124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:29.839915 12124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:29.840154 12124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:29.840747 12124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:29.841650 12124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:29.841701 12124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:29.841740 12124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:29.841766 12124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:29.850481 12124 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:43189
I20260811 22:33:29.850507 12336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:43189 every 8 connection(s)
I20260811 22:33:29.851399 12124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:33:29.853000 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 12124
I20260811 22:33:29.853116 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:33:29.853462 19278 external_mini_cluster.cc:1428] Setting key 9a546a1ba3b7fce32936ccf156d6dced
Aug 11 22:33:29 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487609, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:33:29.865865 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:29.853288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:37175 (local address 127.18.211.190:44989)
0811 22:33:29.853419 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:29.853422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:29.854432 (+ 1010us) server_negotiation.cc:407] Connection header received
0811 22:33:29.855841 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:29.855845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:29.855897 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:29.855998 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:29.857163 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.857727 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.858898 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.859138 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.859258 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:29.861955 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:29.861960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:29.861963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:29.861995 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:29.863707 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:29.864211 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:29.864217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:29.864220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:29.864315 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:29.864765 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:29.864770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:29.864772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:29.865011 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:29.865100 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:29.865644 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 22:33:29.865746 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:33:29.866757 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:29.853773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:37175)
0811 22:33:29.854277 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:29.854329 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:29.855618 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:29.856181 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:29.856188 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:29.856551 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:29.857018 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:29.857031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.857854 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.857857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:29.858767 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:29.858777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:29.859298 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:29.859302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:29.859486 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:29.860124 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:29.860150 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:29.861780 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:29.863826 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:29.863830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:29.863842 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:29.864093 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:29.864420 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:29.864423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:29.864424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:29.864571 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:29.865109 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:29.865114 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:29.865390 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 22:33:29.865547 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:33:29.865918 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":143,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:33:29.867980 12337 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44989
I20260811 22:33:29.868242 12337 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:29.868848 12337 heartbeater.cc:511] Master 127.18.211.190:44989 requested a full tablet report, sending...
I20260811 22:33:29.870113 11794 ts_manager.cc:194] Registered new tserver with Master: 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189)
I20260811 22:33:29.870968 11794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:37175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:33:29.884419 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44989
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:29.998540 12346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:29.998802 12346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:29.998847 12346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:30.002938 12346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:30.003005 12346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:30.003064 12346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:33:30.008458 12346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:30.009513 12346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:30.010550 12346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:33:30.020599 12346 server_base.cc:1034] running on GCE node
W20260811 22:33:30.020572 12352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:30.020548 12355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:30.020779 12351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:30.021178 12346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:30.021867 12346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:30.023057 12346 hybrid_clock.cc:648] HybridClock initialized: now 1786487610023015 us; error 53 us; skew 500 ppm
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:30.034090 12346 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:33:30.035002 12346 webserver.cc:460] Webserver started at http://127.18.211.131:46769/ using document root <none> and password file <none>
I20260811 22:33:30.035555 12346 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:30.035614 12346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:30.035785 12346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:30.037905 12346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "a3106d265a594a869de88bb55cd2391a"
format_stamp: "Formatted at 2026-08-11 22:33:30 on dist-test-slave-8hhm"
server_key: "b6bcad17abb7d7ed0d9e4d33c7229a16"
server_key_iv: "3e2f826c3d9b30183d3dff5ba792eee7"
server_key_version: "encryptionkey@0"
I20260811 22:33:30.038522 12346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "a3106d265a594a869de88bb55cd2391a"
format_stamp: "Formatted at 2026-08-11 22:33:30 on dist-test-slave-8hhm"
server_key: "b6bcad17abb7d7ed0d9e4d33c7229a16"
server_key_iv: "3e2f826c3d9b30183d3dff5ba792eee7"
server_key_version: "encryptionkey@0"
I20260811 22:33:30.042411 12346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:33:30.044787 12367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.046068 12346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:33:30.046211 12346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "a3106d265a594a869de88bb55cd2391a"
format_stamp: "Formatted at 2026-08-11 22:33:30 on dist-test-slave-8hhm"
server_key: "b6bcad17abb7d7ed0d9e4d33c7229a16"
server_key_iv: "3e2f826c3d9b30183d3dff5ba792eee7"
server_key_version: "encryptionkey@0"
I20260811 22:33:30.046319 12346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:30.059394 12346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:30.062693 12346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:30.062889 12346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:30.063499 12346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:30.064320 12346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:30.064368 12346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.064430 12346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:30.064472 12346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.073649 12346 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:44393
I20260811 22:33:30.073719 12555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:44393 every 8 connection(s)
I20260811 22:33:30.074574 12346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:33:30.081800 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 12346
I20260811 22:33:30.081900 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:33:30.082199 19278 external_mini_cluster.cc:1428] Setting key 9c96873d819dfdc727b46719ed08b03c
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:30.088621 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.076587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:51757 (local address 127.18.211.190:44989)
0811 22:33:30.076765 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.076770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.077699 (+ 929us) server_negotiation.cc:407] Connection header received
0811 22:33:30.079025 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.079028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.079076 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.079153 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:30.080292 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.080813 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.082016 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.082245 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.082366 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.085431 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:30.085434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:30.085436 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:30.085460 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:30.086889 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.087350 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.087353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.087354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.087432 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.087724 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.087726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.087727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.087891 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:30.087970 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.088414 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.088506 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:33:30.089471 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.077045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:51757)
0811 22:33:30.077539 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.077582 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.078830 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.079328 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.079335 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.079687 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:30.080156 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.080168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.080921 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.080923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.081881 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.081895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.082515 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.082517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.082710 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.083409 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:30.083432 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:30.085111 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:30.086997 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.087002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.087013 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.087263 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.087509 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.087511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.087512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.087641 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.087990 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:30.087994 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:30.088213 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.088363 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.088594 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":172,"threads_started":1}
I20260811 22:33:30.090595 12556 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44989
I20260811 22:33:30.090818 12556 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:30.091250 12556 heartbeater.cc:511] Master 127.18.211.190:44989 requested a full tablet report, sending...
I20260811 22:33:30.092226 11794 ts_manager.cc:194] Registered new tserver with Master: a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393)
I20260811 22:33:30.092792 11794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:51757
I20260811 22:33:30.096021 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:33:30.104810 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.097981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38570 (local address 127.18.211.190:44989)
0811 22:33:30.098097 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.098100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.098325 (+ 225us) server_negotiation.cc:407] Connection header received
0811 22:33:30.098422 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.098424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.098460 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.098517 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:30.099169 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.099640 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.100472 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.100671 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.100750 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.102014 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:30.102017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:30.102018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:30.102040 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:30.103446 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.103809 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.103811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.103812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.103885 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.104112 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.104114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.104115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.104316 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:30.104421 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.104614 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.104702 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:33:30.107640 11794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:33:30.110692 11794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:33:30.125190 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.119757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.18.211.131:44393)
0811 22:33:30.120353 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.120366 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.120389 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:33:30.120466 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.120471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.120665 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.120831 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:30.121196 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.122046 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.123973 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.124525 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.124664 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.124812 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.124939 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.125032 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":159,"threads_started":1}
I20260811 22:33:30.125243 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.118964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:44393 (local address 127.0.0.1:51396)
0811 22:33:30.119822 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.119837 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.119930 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.120984 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.120987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.121003 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:30.121070 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.121077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.122184 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.122188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.123827 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.123838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.124654 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.124658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.124762 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.124798 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.124912 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.125030 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":65,"thread_start_us":94,"threads_started":1}
I20260811 22:33:30.125312 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.118876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:43189 (local address 127.0.0.1:39996)
0811 22:33:30.119217 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.119265 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.119389 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.120645 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.120651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.120675 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:30.120850 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.120862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.122495 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.122499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.123826 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.123838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.124843 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.124847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.124933 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.124971 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.125028 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.125097 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":122,"thread_start_us":106,"threads_started":1}
I20260811 22:33:30.125552 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.119740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47578 (local address 127.18.211.129:44781)
0811 22:33:30.120168 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.120182 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.120482 (+ 300us) server_negotiation.cc:407] Connection header received
0811 22:33:30.120569 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.120576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.120759 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.120922 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:30.121152 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.122391 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.123857 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.124696 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.124837 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.124968 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.125300 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.125410 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":129,"threads_started":1}
I20260811 22:33:30.125890 12572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.119265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44781 (local address 127.0.0.1:47578)
0811 22:33:30.120287 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.120299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.120389 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.120898 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.120901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.120919 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:30.121024 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.121031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.122544 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.122547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.123733 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.123743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.125035 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.125038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.125116 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.125142 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.125700 (+ 558us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.125774 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":60,"thread_start_us":98,"threads_started":1}
I20260811 22:33:30.125929 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.119595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.18.211.130:43189)
0811 22:33:30.120117 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.120130 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.120157 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:33:30.120243 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.120250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.120474 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.120997 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:30.121152 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.122353 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.123955 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.124711 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.125504 (+ 793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.125642 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.125710 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.125793 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":179,"threads_started":1}
I20260811 22:33:30.128563 11998 tablet_service.cc:1467] Processing CreateTablet for tablet 685efbcaa8384cf193f118d9dfca0065 (DEFAULT_TABLE table=test-table [id=51e37faa7d9b44cdace83a0d14b83feb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:30.128683 12448 tablet_service.cc:1467] Processing CreateTablet for tablet 685efbcaa8384cf193f118d9dfca0065 (DEFAULT_TABLE table=test-table [id=51e37faa7d9b44cdace83a0d14b83feb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:30.128693 12227 tablet_service.cc:1467] Processing CreateTablet for tablet 685efbcaa8384cf193f118d9dfca0065 (DEFAULT_TABLE table=test-table [id=51e37faa7d9b44cdace83a0d14b83feb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:30.129652 11998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685efbcaa8384cf193f118d9dfca0065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:30.129737 12448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685efbcaa8384cf193f118d9dfca0065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:30.129736 12227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685efbcaa8384cf193f118d9dfca0065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:30.136000 12579 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Bootstrap starting.
I20260811 22:33:30.137378 12581 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Bootstrap starting.
I20260811 22:33:30.138602 12579 tablet_bootstrap.cc:654] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:30.139665 12581 tablet_bootstrap.cc:654] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:30.139925 12579 log.cc:824] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:30.140475 12581 log.cc:824] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:30.140789 12582 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Bootstrap starting.
I20260811 22:33:30.142501 12582 tablet_bootstrap.cc:654] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:30.142510 12581 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: No bootstrap required, opened a new log
I20260811 22:33:30.142738 12581 ts_tablet_manager.cc:1397] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:33:30.143597 12582 log.cc:824] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:30.145259 12579 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: No bootstrap required, opened a new log
I20260811 22:33:30.145473 12579 ts_tablet_manager.cc:1397] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.004s
I20260811 22:33:30.145617 12582 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: No bootstrap required, opened a new log
I20260811 22:33:30.145798 12582 ts_tablet_manager.cc:1397] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:33:30.146130 12581 raft_consensus.cc:348] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.146384 12581 raft_consensus.cc:374] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:30.146457 12581 raft_consensus.cc:729] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221d520f4bc144d680a536275a4239d7, State: Initialized, Role: FOLLOWER
I20260811 22:33:30.146935 12581 consensus_queue.cc:260] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.148114 12581 ts_tablet_manager.cc:1428] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:33:30.148277 12337 heartbeater.cc:503] Master 127.18.211.190:44989 was elected leader, sending a full tablet report...
I20260811 22:33:30.149447 12579 raft_consensus.cc:348] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.149557 12582 raft_consensus.cc:348] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.149740 12579 raft_consensus.cc:374] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:30.149760 12582 raft_consensus.cc:374] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:30.149822 12582 raft_consensus.cc:729] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3106d265a594a869de88bb55cd2391a, State: Initialized, Role: FOLLOWER
I20260811 22:33:30.149863 12579 raft_consensus.cc:729] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83205a60f3144ed3b072429678f08ff5, State: Initialized, Role: FOLLOWER
I20260811 22:33:30.150300 12582 consensus_queue.cc:260] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.150300 12579 consensus_queue.cc:260] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.151309 12582 ts_tablet_manager.cc:1428] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:33:30.151433 12114 heartbeater.cc:503] Master 127.18.211.190:44989 was elected leader, sending a full tablet report...
I20260811 22:33:30.151549 12579 ts_tablet_manager.cc:1428] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:33:30.152101 12556 heartbeater.cc:503] Master 127.18.211.190:44989 was elected leader, sending a full tablet report...
I20260811 22:33:30.174786 12591 raft_consensus.cc:482] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:30.175058 12591 raft_consensus.cc:504] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.176357 12591 leader_election.cc:288] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189), 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781)
I20260811 22:33:30.181392 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.176909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:53915 (local address 127.18.211.129:44781)
0811 22:33:30.177049 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.177054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.177293 (+ 239us) server_negotiation.cc:407] Connection header received
0811 22:33:30.177409 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.177411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.177458 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.177539 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:30.177841 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.178615 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.180027 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.180645 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.180787 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.180840 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.181154 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.181260 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:33:30.181392 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.176810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:49591 (local address 127.18.211.130:43189)
0811 22:33:30.176947 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.176951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.176967 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:33:30.177132 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.177136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.177211 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.177313 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:30.177800 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.178615 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.180070 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.180667 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.180787 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.180840 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.181154 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.181265 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:33:30.181396 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.176632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:43189 (local address 127.18.211.131:49591)
0811 22:33:30.176868 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.176885 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.176991 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.177466 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.177473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.177488 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:30.177649 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.177657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.178744 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.178750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.179894 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.179904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.180793 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.180796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.180974 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.181033 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.181114 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.181231 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":105}
I20260811 22:33:30.181411 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.176652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44781 (local address 127.18.211.131:53915)
0811 22:33:30.177178 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.177192 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.177317 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.177577 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.177579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.177591 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:30.177716 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.177726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.178790 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.178793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.179892 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.179903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.180793 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.180796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.180955 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.181039 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.181114 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.181227 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":171,"thread_start_us":148,"threads_started":1}
I20260811 22:33:30.182107 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685efbcaa8384cf193f118d9dfca0065" candidate_uuid: "a3106d265a594a869de88bb55cd2391a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d520f4bc144d680a536275a4239d7" is_pre_election: true
I20260811 22:33:30.182185 12038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685efbcaa8384cf193f118d9dfca0065" candidate_uuid: "a3106d265a594a869de88bb55cd2391a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83205a60f3144ed3b072429678f08ff5" is_pre_election: true
I20260811 22:33:30.182467 12257 raft_consensus.cc:2454] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3106d265a594a869de88bb55cd2391a in term 0.
I20260811 22:33:30.182482 12038 raft_consensus.cc:2454] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3106d265a594a869de88bb55cd2391a in term 0.
I20260811 22:33:30.182973 12373 leader_election.cc:302] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a; no voters:
I20260811 22:33:30.183281 12591 raft_consensus.cc:2781] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:33:30.183372 12591 raft_consensus.cc:482] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:33:30.183416 12591 raft_consensus.cc:3037] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:30.184443 12591 raft_consensus.cc:504] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.184801 12591 leader_election.cc:288] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [CANDIDATE]: Term 1 election: Requested vote from peers 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189), 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781)
I20260811 22:33:30.185119 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685efbcaa8384cf193f118d9dfca0065" candidate_uuid: "a3106d265a594a869de88bb55cd2391a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d520f4bc144d680a536275a4239d7"
I20260811 22:33:30.185187 12038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685efbcaa8384cf193f118d9dfca0065" candidate_uuid: "a3106d265a594a869de88bb55cd2391a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83205a60f3144ed3b072429678f08ff5"
I20260811 22:33:30.185277 12257 raft_consensus.cc:3037] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:30.185330 12038 raft_consensus.cc:3037] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:30.186662 12038 raft_consensus.cc:2454] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3106d265a594a869de88bb55cd2391a in term 1.
I20260811 22:33:30.186662 12257 raft_consensus.cc:2454] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3106d265a594a869de88bb55cd2391a in term 1.
I20260811 22:33:30.186957 12373 leader_election.cc:302] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a; no voters:
I20260811 22:33:30.187117 12591 raft_consensus.cc:2781] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:30.187354 12591 raft_consensus.cc:686] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 1 LEADER]: Becoming Leader. State: Replica: a3106d265a594a869de88bb55cd2391a, State: Running, Role: LEADER
I20260811 22:33:30.187767 12591 consensus_queue.cc:237] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.191056 11794 catalog_manager.cc:5495] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a reported cstate change: term changed from 0 to 1, leader changed from <none> to a3106d265a594a869de88bb55cd2391a (127.18.211.131). New cstate: current_term: 1 leader_uuid: "a3106d265a594a869de88bb55cd2391a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } health_report { overall_health: UNKNOWN } } }
I20260811 22:33:30.202960 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.199318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.18.211.131:44393)
0811 22:33:30.199440 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.199447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.199482 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:33:30.199680 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.199683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.199744 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.199857 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:33:30.200101 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.200596 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.201532 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.201754 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.201866 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.202168 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:33:30.202573 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:33:30.202725 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.202761 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.202816 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:33:30.208781 12038 raft_consensus.cc:1261] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Refusing update from remote peer a3106d265a594a869de88bb55cd2391a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:30.208782 12257 raft_consensus.cc:1261] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 1 FOLLOWER]: Refusing update from remote peer a3106d265a594a869de88bb55cd2391a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:30.209538 12593 consensus_queue.cc:1035] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [LEADER]: Connected to new peer: Peer: permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:30.209705 12591 consensus_queue.cc:1035] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [LEADER]: Connected to new peer: Peer: permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:30.217428 12598 mvcc.cc:205] Tried to move back new op lower bound from 7317453251407544320 to 7317453251331792896. Current Snapshot: MvccSnapshot[applied={T|T < 7317453251407544320}]
I20260811 22:33:30.218420 12596 mvcc.cc:205] Tried to move back new op lower bound from 7317453251407544320 to 7317453251331792896. Current Snapshot: MvccSnapshot[applied={T|T < 7317453251407544320}]
I20260811 22:33:30.226019 11793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:30.226215 11793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:30.228641 11793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Sending DeleteTablet for 3 replicas of tablet 685efbcaa8384cf193f118d9dfca0065
I20260811 22:33:30.229455 12227 tablet_service.cc:1514] Processing DeleteTablet for tablet 685efbcaa8384cf193f118d9dfca0065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:30 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:39996
I20260811 22:33:30.229451 12448 tablet_service.cc:1514] Processing DeleteTablet for tablet 685efbcaa8384cf193f118d9dfca0065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:30 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:51396
I20260811 22:33:30.229983 12619 tablet_replica.cc:330] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: stopping tablet replica
I20260811 22:33:30.229969 11998 tablet_service.cc:1514] Processing DeleteTablet for tablet 685efbcaa8384cf193f118d9dfca0065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:30 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:47578
I20260811 22:33:30.230315 12619 raft_consensus.cc:2229] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:33:30.230499 12621 tablet_replica.cc:330] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: stopping tablet replica
I20260811 22:33:30.230670 12619 raft_consensus.cc:2258] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:30.230815 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 11888
I20260811 22:33:30.230937 12621 raft_consensus.cc:2229] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:30.231045 12620 tablet_replica.cc:330] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: stopping tablet replica
I20260811 22:33:30.231227 12621 raft_consensus.cc:2258] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:30.234993 12619 ts_tablet_manager.cc:1905] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:30.237892 12619 ts_tablet_manager.cc:1918] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:33:30.237985 12619 log.cc:1196] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/685efbcaa8384cf193f118d9dfca0065
I20260811 22:33:30.238346 12619 ts_tablet_manager.cc:1939] T 685efbcaa8384cf193f118d9dfca0065 P a3106d265a594a869de88bb55cd2391a: Deleting consensus metadata
I20260811 22:33:30.233918 12620 raft_consensus.cc:2229] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:30.238917 12620 raft_consensus.cc:2258] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:30.240592 12620 ts_tablet_manager.cc:1905] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:30.243995 12620 ts_tablet_manager.cc:1918] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:33:30.244102 12620 log.cc:1196] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/685efbcaa8384cf193f118d9dfca0065
I20260811 22:33:30.244455 12620 ts_tablet_manager.cc:1939] T 685efbcaa8384cf193f118d9dfca0065 P 221d520f4bc144d680a536275a4239d7: Deleting consensus metadata
I20260811 22:33:30.244407 11772 catalog_manager.cc:4841] TS a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393): tablet 685efbcaa8384cf193f118d9dfca0065 (table test-table [id=51e37faa7d9b44cdace83a0d14b83feb]) successfully deleted
I20260811 22:33:30.245733 11776 catalog_manager.cc:4841] TS 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189): tablet 685efbcaa8384cf193f118d9dfca0065 (table test-table [id=51e37faa7d9b44cdace83a0d14b83feb]) successfully deleted
W20260811 22:33:30.247828 11773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:44781 (error 108)
I20260811 22:33:30.248764 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 12124
I20260811 22:33:30.265708 12572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.248658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44781 (local address 127.0.0.1:47586)
0811 22:33:30.248796 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.248887 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:44781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 22:33:30.266083 11773 catalog_manager.cc:4584] TS 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 685efbcaa8384cf193f118d9dfca0065: Network error: Client connection negotiation failed: client connection to 127.18.211.129:44781: connect: Connection refused (error 111)
I20260811 22:33:30.271646 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 12346
I20260811 22:33:30.287532 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 11745
I20260811 22:33:30.300347 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44989
--webserver_interface=127.18.211.190
--webserver_port=39345
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:44989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:30.416049 12625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:30.416384 12625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:30.416471 12625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:30.420544 12625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:30.420635 12625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:30.420675 12625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:30.420742 12625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:30.420782 12625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:30.427317 12625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:44989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=39345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:30.428485 12625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:30.429646 12625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:30.445235 12650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:30.445247 12646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:30.445253 12647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:30.445384 12625 server_base.cc:1034] running on GCE node
I20260811 22:33:30.446312 12625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:30.447597 12625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:30.448796 12625 hybrid_clock.cc:648] HybridClock initialized: now 1786487610448768 us; error 38 us; skew 500 ppm
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:30.451808 12625 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:30.452780 12625 webserver.cc:460] Webserver started at http://127.18.211.190:39345/ using document root <none> and password file <none>
I20260811 22:33:30.453476 12625 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:30.453588 12625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:30.457670 12625 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:33:30.459914 12662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.461185 12625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:33:30.461392 12625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d02403a59d954c6bb82894ce851b2155"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "7ebedd3b2db3cdfc5da96369ce8da45f"
server_key_iv: "c40a164f44375289870e06f822b42d7d"
server_key_version: "encryptionkey@0"
I20260811 22:33:30.462031 12625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:30.471006 12625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:30.473943 12625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:30.474153 12625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:30.481438 12625 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:44989
I20260811 22:33:30.481734 12743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:44989 every 8 connection(s)
I20260811 22:33:30.482580 12625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:30.487048 12748 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:33:30.487797 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 12625
I20260811 22:33:30.488334 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:44781
--local_ip_for_outbound_sockets=127.18.211.129
--tserver_master_addrs=127.18.211.190:44989
--webserver_port=34943
--webserver_interface=127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:30.489353 12748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Bootstrap starting.
I20260811 22:33:30.507241 12748 log.cc:824] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:30.516150 12748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:33:30.516536 12748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Bootstrap complete.
I20260811 22:33:30.520632 12748 raft_consensus.cc:348] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:30.521031 12748 raft_consensus.cc:729] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02403a59d954c6bb82894ce851b2155, State: Initialized, Role: FOLLOWER
I20260811 22:33:30.521584 12748 consensus_queue.cc:260] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:30.521713 12748 raft_consensus.cc:388] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:33:30.521813 12748 raft_consensus.cc:482] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:33:30.521943 12748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:33:30.523903 12748 raft_consensus.cc:504] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:30.524266 12748 leader_election.cc:302] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d02403a59d954c6bb82894ce851b2155; no voters:
I20260811 22:33:30.524613 12748 leader_election.cc:288] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:33:30.524698 12756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:33:30.524930 12756 raft_consensus.cc:686] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 LEADER]: Becoming Leader. State: Replica: d02403a59d954c6bb82894ce851b2155, State: Running, Role: LEADER
I20260811 22:33:30.525336 12756 consensus_queue.cc:237] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:30.525930 12748 sys_catalog.cc:564] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:30.526854 12759 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d02403a59d954c6bb82894ce851b2155. Latest consensus state: current_term: 2 leader_uuid: "d02403a59d954c6bb82894ce851b2155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } } }
I20260811 22:33:30.526986 12759 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:30.527298 12757 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d02403a59d954c6bb82894ce851b2155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } } }
I20260811 22:33:30.527391 12757 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:30.530793 12776 catalog_manager.cc:1259] Loaded cluster ID: a6edb518b12d45f5ba3aa3dee6103bae
I20260811 22:33:30.530879 12776 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: loading cluster ID for follower catalog manager: success
W20260811 22:33:30.531729 12776 catalog_manager.cc:1576] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260811 22:33:30.531817 12776 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260811 22:33:30.531893 12769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:30.534092 12769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:30.534523 12769 catalog_manager.cc:1259] Loaded cluster ID: a6edb518b12d45f5ba3aa3dee6103bae
I20260811 22:33:30.534603 12769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:33:30.535477 12769 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 22:33:30.538239 12769 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 22:33:30.542744 12769 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:33:30.543459 12769 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Loaded TSK: 0
I20260811 22:33:30.544497 12769 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:33:30.614188 12750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:30.614482 12750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:30.614557 12750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:30.618880 12750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:30.618970 12750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:30.619055 12750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:33:30.624212 12750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:44781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=34943
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:30.625274 12750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:30.626442 12750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:30.636896 12795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:30.636934 12791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:30.636907 12792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:30.637498 12750 server_base.cc:1034] running on GCE node
I20260811 22:33:30.637964 12750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:30.638610 12750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:30.639810 12750 hybrid_clock.cc:648] HybridClock initialized: now 1786487610639778 us; error 53 us; skew 500 ppm
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:30.642820 12750 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:33:30.643734 12750 webserver.cc:460] Webserver started at http://127.18.211.129:34943/ using document root <none> and password file <none>
I20260811 22:33:30.644297 12750 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:30.644375 12750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:30.648456 12750 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:33:30.651332 12803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.652697 12750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:33:30.652840 12750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "83205a60f3144ed3b072429678f08ff5"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "196a9d0706b88c1a66c9bf977329268f"
server_key_iv: "d5acff0820ee6b9fa326a20d67368d34"
server_key_version: "encryptionkey@0"
I20260811 22:33:30.653527 12750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:30.664947 12750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:30.668237 12750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:30.668442 12750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:30.669036 12750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:30.670267 12815 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:33:30.674489 12750 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:33:30.674553 12750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:33:30.674619 12750 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:33:30.676295 12750 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:33:30.676362 12750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 22:33:30.676716 12815 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Bootstrap starting.
I20260811 22:33:30.687580 12750 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:44781
I20260811 22:33:30.687888 12998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:44781 every 8 connection(s)
I20260811 22:33:30.688901 12750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:33:30.697018 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 12750
I20260811 22:33:30.697582 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:43189
--local_ip_for_outbound_sockets=127.18.211.130
--tserver_master_addrs=127.18.211.190:44989
--webserver_port=33563
--webserver_interface=127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:30.697901 12815 log.cc:824] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:30.706509 12815 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:33:30.706979 12815 tablet_bootstrap.cc:492] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Bootstrap complete.
I20260811 22:33:30.708405 12815 ts_tablet_manager.cc:1397] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.011s
I20260811 22:33:30.708719 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.691251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:60423)
0811 22:33:30.691882 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.691924 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.692770 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.694011 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.694019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.694484 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:30.694954 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.694968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.696337 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.696340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.697494 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.697505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.698723 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.698727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.698935 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.699880 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:30.699901 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:30.701782 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:30.704735 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.704741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.704754 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.705112 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.705609 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.705612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.705614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.705799 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.706732 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:30.706740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:30.707061 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.707273 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.707640 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":141,"threads_started":1}
I20260811 22:33:30.709113 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.691222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:60423 (local address 127.18.211.190:44989)
0811 22:33:30.691608 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.691623 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.692061 (+ 438us) server_negotiation.cc:407] Connection header received
0811 22:33:30.693205 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.693278 (+ 73us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.693605 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.693949 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:30.695143 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.696159 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.697631 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.698294 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.698490 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.702004 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:30.702013 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:30.702034 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:30.702076 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:30.704580 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.705286 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.705296 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.705303 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.705485 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.705918 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.705925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.705928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.706489 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:30.706680 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.707445 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.707823 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":94,"threads_started":1}
I20260811 22:33:30.711066 12815 raft_consensus.cc:348] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.711733 12815 raft_consensus.cc:729] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83205a60f3144ed3b072429678f08ff5, State: Initialized, Role: FOLLOWER
I20260811 22:33:30.712050 13000 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44989
I20260811 22:33:30.712268 13000 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:30.712260 12815 consensus_queue.cc:260] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:30.712874 13000 heartbeater.cc:511] Master 127.18.211.190:44989 requested a full tablet report, sending...
I20260811 22:33:30.713347 12815 ts_tablet_manager.cc:1428] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:33:30.715556 12686 ts_manager.cc:194] Registered new tserver with Master: 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781)
I20260811 22:33:30.719022 12686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:60423
I20260811 22:33:30.721884 13000 heartbeater.cc:503] Master 127.18.211.190:44989 was elected leader, sending a full tablet report...
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:33:30.729696 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.718614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47602 (local address 127.18.211.129:44781)
0811 22:33:30.718997 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.719010 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.719433 (+ 423us) server_negotiation.cc:407] Connection header received
0811 22:33:30.719625 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.719630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.719841 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.719977 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:30.720633 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.721459 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.722441 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.722963 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.723080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.725705 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:30.725715 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:30.725730 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:30.725762 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:30.727657 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.728211 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.728225 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.728226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.728357 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.728668 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.728670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.728671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.728923 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:30.729028 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.729420 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.729560 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":116,"threads_started":1}
I20260811 22:33:30.729732 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.718517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44781 (local address 127.0.0.1:47602)
0811 22:33:30.719272 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.719319 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.719465 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.719991 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.719998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.720396 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:30.720501 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.720508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.721579 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.721583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.722339 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.722354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.723100 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.723104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.723235 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.723927 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:30.723939 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:30.725559 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:30.727783 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.727791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.727797 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.728038 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.728454 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.728457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.728460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.728568 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.729039 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:30.729046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:30.729174 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.729346 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.729545 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":105,"threads_started":1}
I20260811 22:33:30.730857 12886 tablet_service.cc:1514] Processing DeleteTablet for tablet 685efbcaa8384cf193f118d9dfca0065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:30 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:47602
I20260811 22:33:30.731717 13019 tablet_replica.cc:330] stopping tablet replica
I20260811 22:33:30.732014 13019 raft_consensus.cc:2229] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:30.732287 13019 raft_consensus.cc:2258] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:30.733312 13019 ts_tablet_manager.cc:1905] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:30.736843 13019 ts_tablet_manager.cc:1918] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:33:30.736967 13019 log.cc:1196] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/685efbcaa8384cf193f118d9dfca0065
I20260811 22:33:30.737322 13019 ts_tablet_manager.cc:1939] T 685efbcaa8384cf193f118d9dfca0065 P 83205a60f3144ed3b072429678f08ff5: Deleting consensus metadata
I20260811 22:33:30.739089 12665 catalog_manager.cc:4841] TS 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781): tablet 685efbcaa8384cf193f118d9dfca0065 (table test-table [id=51e37faa7d9b44cdace83a0d14b83feb]) successfully deleted
W20260811 22:33:30.821801 13011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:30.822089 13011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:30.822132 13011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:30.826246 13011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:30.826304 13011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:30.826406 13011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:33:30.831815 13011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:43189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=33563
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:30.832864 13011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:30.833922 13011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:30.843953 13035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:30.844007 13030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:30.844116 13011 server_base.cc:1034] running on GCE node
W20260811 22:33:30.844018 13029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:30.844497 13011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:30.845209 13011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:30.846439 13011 hybrid_clock.cc:648] HybridClock initialized: now 1786487610846401 us; error 49 us; skew 500 ppm
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:30.849155 13011 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:33:30.850118 13011 webserver.cc:460] Webserver started at http://127.18.211.130:33563/ using document root <none> and password file <none>
I20260811 22:33:30.850687 13011 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:30.850816 13011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:30.854709 13011 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:33:30.856822 13044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.858117 13011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:33:30.858261 13011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "221d520f4bc144d680a536275a4239d7"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "b07e4031899dd6c9031ce6db7cfcf6c7"
server_key_iv: "4278be1ae00f5455a7f956ff5a7241a9"
server_key_version: "encryptionkey@0"
I20260811 22:33:30.858824 13011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:30.878014 13011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:30.881071 13011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:30.881331 13011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:30.882167 13011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:30.883395 13011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:30.883466 13011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.883533 13011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:30.883571 13011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:30.896867 13011 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:43189
I20260811 22:33:30.896935 13231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:43189 every 8 connection(s)
I20260811 22:33:30.898080 13011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:33:30.902618 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 13011
I20260811 22:33:30.903064 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:44393
--local_ip_for_outbound_sockets=127.18.211.131
--tserver_master_addrs=127.18.211.190:44989
--webserver_port=46769
--webserver_interface=127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:33:30 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487610, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:30.913883 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.900182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:36591 (local address 127.18.211.190:44989)
0811 22:33:30.900323 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:30.900327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:30.901114 (+ 787us) server_negotiation.cc:407] Connection header received
0811 22:33:30.902641 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:30.902644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:30.902710 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:30.902821 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:30.904136 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.904714 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.905986 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.906396 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.906505 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.909947 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:30.909951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:30.909954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:30.909987 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:30.911654 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.912203 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.912206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.912208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.912296 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:30.912661 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:30.912664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:30.912666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:30.912865 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:30.912975 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:30.913560 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 22:33:30.913703 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:33:30.914722 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:30.900491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:36591)
0811 22:33:30.900959 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:30.901003 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:30.902423 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:30.903014 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:30.903021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:30.903479 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:30.903980 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.903994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.904883 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.904886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.905855 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:30.905866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:30.906565 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:30.906572 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:30.906810 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:30.907556 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:30.907582 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:30.909739 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:30.911780 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.911786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.911802 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.912079 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.912393 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:30.912397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:30.912399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:30.912552 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:30.912975 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:30.912983 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:30.913308 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 22:33:30.913501 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 22:33:30.913854 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":134,"threads_started":1}
I20260811 22:33:30.916076 13234 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44989
I20260811 22:33:30.916386 13234 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:30.916993 13234 heartbeater.cc:511] Master 127.18.211.190:44989 requested a full tablet report, sending...
I20260811 22:33:30.918200 12686 ts_manager.cc:194] Registered new tserver with Master: 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189)
I20260811 22:33:30.918983 12686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:36591
W20260811 22:33:31.021457 13242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:31.021714 13242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:31.021757 13242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:31.025717 13242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:31.025772 13242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:31.025830 13242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:33:31.031277 13242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:44393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=46769
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:31.032284 13242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:31.033337 13242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:31.043147 13253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:31.043143 13251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:31.043327 13242 server_base.cc:1034] running on GCE node
W20260811 22:33:31.043475 13257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:31.043840 13242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:31.044584 13242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:31.045775 13242 hybrid_clock.cc:648] HybridClock initialized: now 1786487611045758 us; error 26 us; skew 500 ppm
Aug 11 22:33:31 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:31.048494 13242 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:33:31.049491 13242 webserver.cc:460] Webserver started at http://127.18.211.131:46769/ using document root <none> and password file <none>
I20260811 22:33:31.050071 13242 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:31.050187 13242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:31.054234 13242 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:33:31.056340 13269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.057718 13242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:33:31.057857 13242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "a3106d265a594a869de88bb55cd2391a"
format_stamp: "Formatted at 2026-08-11 22:33:30 on dist-test-slave-8hhm"
server_key: "b6bcad17abb7d7ed0d9e4d33c7229a16"
server_key_iv: "3e2f826c3d9b30183d3dff5ba792eee7"
server_key_version: "encryptionkey@0"
I20260811 22:33:31.058432 13242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:31.069054 13242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:31.071995 13242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:31.072203 13242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:31.072794 13242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:31.073729 13242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:31.073800 13242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.073873 13242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:31.073918 13242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.082970 13242 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:44393
I20260811 22:33:31.082990 13466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:44393 every 8 connection(s)
I20260811 22:33:31.083998 13242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:33:31.091647 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 13242
Aug 11 22:33:31 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:31.098107 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.085906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:42967 (local address 127.18.211.190:44989)
0811 22:33:31.086050 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.086054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.086898 (+ 844us) server_negotiation.cc:407] Connection header received
0811 22:33:31.088279 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.088283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.088342 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.088456 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:31.089668 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.090228 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.091378 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.091603 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.091728 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.094245 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:31.094249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:31.094256 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:31.094278 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:31.095882 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:31.096407 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:31.096410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:31.096412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:31.096499 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:31.096901 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:31.096905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:31.096907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:31.097085 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:31.097196 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.097725 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.097943 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":8832}
I20260811 22:33:31.098802 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.086289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:42967)
0811 22:33:31.086745 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.086787 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:31.088054 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:31.088639 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:31.088646 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:31.089004 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:31.089498 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.089511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.090355 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.090363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.091249 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.091258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.091774 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.091778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.091944 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.092600 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:31.092624 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:31.094076 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:31.095999 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:31.096004 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:31.096026 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:31.096299 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:31.096592 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:31.096597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:31.096599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:31.096804 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:31.097201 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:31.097227 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:31.097465 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 22:33:31.097652 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 22:33:31.097980 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":123,"threads_started":1}
I20260811 22:33:31.100085 13467 heartbeater.cc:347] Connected to a master server at 127.18.211.190:44989
I20260811 22:33:31.100349 13467 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:31.100862 13467 heartbeater.cc:511] Master 127.18.211.190:44989 requested a full tablet report, sending...
I20260811 22:33:31.101593 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.092399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.18.211.190:44989)
0811 22:33:31.092705 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.092710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.092830 (+ 120us) server_negotiation.cc:407] Connection header received
0811 22:33:31.092992 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.092995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.093043 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.093137 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:31.093958 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.094473 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.095800 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.096018 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.096145 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.097645 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:31.097650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:31.097652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:31.097713 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:31.099801 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:31.100291 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:31.100295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:31.100298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:31.100383 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:31.100697 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:31.100701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:31.100702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:31.100879 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:31.101092 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.101298 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.101421 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"spinlock_wait_cycles":768,"thread_start_us":106,"threads_started":1}
I20260811 22:33:31.102052 12686 ts_manager.cc:194] Registered new tserver with Master: a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393)
I20260811 22:33:31.102748 12686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:42967
I20260811 22:33:31.103349 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:33:31.107455 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.103968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.18.211.190:44989)
0811 22:33:31.104078 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.104081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.104326 (+ 245us) server_negotiation.cc:407] Connection header received
0811 22:33:31.104424 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.104427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.104535 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.104641 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:33:31.104845 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.105442 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.106311 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.106520 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.106597 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.106857 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:33:31.107175 (+ 318us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:33:31.107270 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.107324 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.107368 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:33:31.108058 12685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:33:31.111395 12685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:33:31.119130 12886 tablet_service.cc:1467] Processing CreateTablet for tablet 51c1216a4edf44319a206ccf54a2227a (DEFAULT_TABLE table=test-table [id=8e2a64b8c34945b59c96757ae160763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:31.119836 12886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51c1216a4edf44319a206ccf54a2227a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:31.122778 12815 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Bootstrap starting.
I20260811 22:33:31.123692 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.118031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:44393 (local address 127.0.0.1:51410)
0811 22:33:31.118153 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.118170 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:31.118262 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:31.119516 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:31.119520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:31.119541 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:31.119644 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.119651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.121166 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.121170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.122185 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.122208 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.123252 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.123256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.123348 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.123388 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:33:31.123476 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:33:31.123559 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:33:31.123719 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.118688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.18.211.131:44393)
0811 22:33:31.118989 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.119001 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.119026 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:33:31.119104 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.119109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.119327 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.119820 (+ 493us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:31.119966 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.121031 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.122336 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.123109 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.123266 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.123402 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.123505 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.123593 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":126,"threads_started":1}
I20260811 22:33:31.123730 12815 tablet_bootstrap.cc:654] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:31.124207 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.118351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:43189 (local address 127.0.0.1:40006)
0811 22:33:31.118613 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.118627 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:31.118739 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:31.120188 (+ 1449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:31.120192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:31.120212 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:31.120296 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.120303 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.121829 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.121833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.122763 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.122773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.123909 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.123914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.123992 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.124011 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:33:31.124059 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 22:33:31.124108 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":114,"threads_started":1}
I20260811 22:33:31.124388 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.119248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40006 (local address 127.18.211.130:43189)
0811 22:33:31.119615 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.119628 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.119678 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:33:31.119762 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.119767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.120024 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.120181 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:31.120450 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.121670 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.122914 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.123734 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.123928 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.124076 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.124137 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.124238 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":137,"threads_started":1}
I20260811 22:33:31.126170 12815 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: No bootstrap required, opened a new log
I20260811 22:33:31.126302 12815 ts_tablet_manager.cc:1397] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:33:31.126765 12815 raft_consensus.cc:348] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.127084 12815 raft_consensus.cc:374] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:31.127208 12815 raft_consensus.cc:729] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83205a60f3144ed3b072429678f08ff5, State: Initialized, Role: FOLLOWER
I20260811 22:33:31.127414 12815 consensus_queue.cc:260] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.127813 12815 ts_tablet_manager.cc:1428] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:33:31.127647 13347 tablet_service.cc:1467] Processing CreateTablet for tablet 51c1216a4edf44319a206ccf54a2227a (DEFAULT_TABLE table=test-table [id=8e2a64b8c34945b59c96757ae160763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:31.127647 13129 tablet_service.cc:1467] Processing CreateTablet for tablet 51c1216a4edf44319a206ccf54a2227a (DEFAULT_TABLE table=test-table [id=8e2a64b8c34945b59c96757ae160763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:31.128808 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51c1216a4edf44319a206ccf54a2227a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:31.128811 13347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51c1216a4edf44319a206ccf54a2227a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:31.137120 13483 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Bootstrap starting.
I20260811 22:33:31.137606 13487 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Bootstrap starting.
I20260811 22:33:31.140360 13487 tablet_bootstrap.cc:654] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:31.140360 13483 tablet_bootstrap.cc:654] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:31.141319 13483 log.cc:824] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:31.141319 13487 log.cc:824] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:31.143080 13487 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: No bootstrap required, opened a new log
I20260811 22:33:31.143096 13483 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: No bootstrap required, opened a new log
I20260811 22:33:31.143290 13483 ts_tablet_manager.cc:1397] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:33:31.143290 13487 ts_tablet_manager.cc:1397] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:33:31.146432 13487 raft_consensus.cc:348] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.146432 13483 raft_consensus.cc:348] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.146642 13487 raft_consensus.cc:374] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:31.146677 13483 raft_consensus.cc:374] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:31.146703 13487 raft_consensus.cc:729] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221d520f4bc144d680a536275a4239d7, State: Initialized, Role: FOLLOWER
I20260811 22:33:31.146767 13483 raft_consensus.cc:729] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3106d265a594a869de88bb55cd2391a, State: Initialized, Role: FOLLOWER
I20260811 22:33:31.147178 13483 consensus_queue.cc:260] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.147178 13487 consensus_queue.cc:260] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.147873 13467 heartbeater.cc:503] Master 127.18.211.190:44989 was elected leader, sending a full tablet report...
I20260811 22:33:31.148051 13234 heartbeater.cc:503] Master 127.18.211.190:44989 was elected leader, sending a full tablet report...
I20260811 22:33:31.148141 13487 ts_tablet_manager.cc:1428] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:33:31.148138 13483 ts_tablet_manager.cc:1428] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 22:33:31.149123 13235 tablet.cc:2369] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:33:31.190392 13004 tablet.cc:2369] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:33:31.200655 13492 raft_consensus.cc:482] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:31.200915 13492 raft_consensus.cc:504] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.202186 13492 leader_election.cc:288] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393), 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781)
I20260811 22:33:31.206809 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.202445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:44393 (local address 127.18.211.130:47499)
0811 22:33:31.202692 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.202709 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:31.202810 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:31.203153 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:31.203156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:31.203178 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:31.203304 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.203313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.204306 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.204309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.205589 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.205600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.206392 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.206395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.206480 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.206501 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:33:31.206580 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:33:31.206633 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":29}
I20260811 22:33:31.206840 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.202720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:47499 (local address 127.18.211.131:44393)
0811 22:33:31.202880 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.202885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.202901 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:33:31.202963 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.202968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.203027 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.203120 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:31.203436 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.204186 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.205719 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.206268 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.206392 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.206447 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.206615 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.206715 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:33:31.207325 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.202445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44781 (local address 127.18.211.130:41019)
0811 22:33:31.203288 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.203301 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:31.203404 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:31.203689 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:31.203691 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:31.203706 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:31.203794 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.203798 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.204670 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.204673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.205805 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:31.205812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.206742 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.206746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:31.206842 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.207021 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 22:33:31.207120 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:33:31.207193 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":396,"thread_start_us":152,"threads_started":1}
I20260811 22:33:31.207399 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.202656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:41019 (local address 127.18.211.129:44781)
0811 22:33:31.202821 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.202827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.203414 (+ 587us) server_negotiation.cc:407] Connection header received
0811 22:33:31.203499 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.203502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.203565 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.203672 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:31.203911 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.204552 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.205921 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.206608 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.206728 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.206888 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.207168 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.207250 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:33:31.207650 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c1216a4edf44319a206ccf54a2227a" candidate_uuid: "221d520f4bc144d680a536275a4239d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3106d265a594a869de88bb55cd2391a" is_pre_election: true
I20260811 22:33:31.207940 12922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c1216a4edf44319a206ccf54a2227a" candidate_uuid: "221d520f4bc144d680a536275a4239d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83205a60f3144ed3b072429678f08ff5" is_pre_election: true
I20260811 22:33:31.207991 13379 raft_consensus.cc:2454] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 221d520f4bc144d680a536275a4239d7 in term 0.
I20260811 22:33:31.208243 12922 raft_consensus.cc:2454] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 221d520f4bc144d680a536275a4239d7 in term 0.
I20260811 22:33:31.208519 13046 leader_election.cc:302] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a; no voters:
I20260811 22:33:31.208838 13492 raft_consensus.cc:2781] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:33:31.208940 13492 raft_consensus.cc:482] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:33:31.209005 13492 raft_consensus.cc:3037] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:31.210115 13492 raft_consensus.cc:504] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.210484 13492 leader_election.cc:288] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [CANDIDATE]: Term 1 election: Requested vote from peers a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393), 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781)
I20260811 22:33:31.210839 12922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c1216a4edf44319a206ccf54a2227a" candidate_uuid: "221d520f4bc144d680a536275a4239d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83205a60f3144ed3b072429678f08ff5"
I20260811 22:33:31.210862 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c1216a4edf44319a206ccf54a2227a" candidate_uuid: "221d520f4bc144d680a536275a4239d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3106d265a594a869de88bb55cd2391a"
I20260811 22:33:31.210981 12922 raft_consensus.cc:3037] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:31.210966 13379 raft_consensus.cc:3037] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:31.211997 12922 raft_consensus.cc:2454] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 221d520f4bc144d680a536275a4239d7 in term 1.
I20260811 22:33:31.212082 13379 raft_consensus.cc:2454] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 221d520f4bc144d680a536275a4239d7 in term 1.
I20260811 22:33:31.212359 13047 leader_election.cc:302] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d520f4bc144d680a536275a4239d7, 83205a60f3144ed3b072429678f08ff5; no voters:
I20260811 22:33:31.212572 13492 raft_consensus.cc:2781] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:31.212837 13492 raft_consensus.cc:686] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 1 LEADER]: Becoming Leader. State: Replica: 221d520f4bc144d680a536275a4239d7, State: Running, Role: LEADER
I20260811 22:33:31.213200 13492 consensus_queue.cc:237] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.216776 12686 catalog_manager.cc:5495] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 221d520f4bc144d680a536275a4239d7 (127.18.211.130). New cstate: current_term: 1 leader_uuid: "221d520f4bc144d680a536275a4239d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } health_report { overall_health: UNKNOWN } } }
I20260811 22:33:31.253556 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.249728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40010 (local address 127.18.211.130:43189)
0811 22:33:31.249863 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:31.249870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:31.249886 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:33:31.250092 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:31.250094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:31.250152 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:31.250278 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:33:31.250528 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.251122 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.251971 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:31.252261 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:31.252346 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:31.252663 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:33:31.253084 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:33:31.253177 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:31.253331 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 22:33:31.253381 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:33:31.259637 13379 raft_consensus.cc:1261] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 1 FOLLOWER]: Refusing update from remote peer 221d520f4bc144d680a536275a4239d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:31.259634 12922 raft_consensus.cc:1261] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Refusing update from remote peer 221d520f4bc144d680a536275a4239d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:31.260346 13492 consensus_queue.cc:1035] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:31.260609 13494 consensus_queue.cc:1035] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:31.269492 13500 mvcc.cc:205] Tried to move back new op lower bound from 7317453255712014336 to 7317453255531999232. Current Snapshot: MvccSnapshot[applied={T|T < 7317453255712014336}]
I20260811 22:33:31.275454 12685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:31.275599 12685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:31.277998 12685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Sending DeleteTablet for 3 replicas of tablet 51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:31.278682 13347 tablet_service.cc:1514] Processing DeleteTablet for tablet 51c1216a4edf44319a206ccf54a2227a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:31 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:51410
I20260811 22:33:31.278679 13129 tablet_service.cc:1514] Processing DeleteTablet for tablet 51c1216a4edf44319a206ccf54a2227a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:31 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:40006
I20260811 22:33:31.278837 12886 tablet_service.cc:1514] Processing DeleteTablet for tablet 51c1216a4edf44319a206ccf54a2227a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:31 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:47602
I20260811 22:33:31.279201 13520 tablet_replica.cc:330] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: stopping tablet replica
I20260811 22:33:31.279214 13523 tablet_replica.cc:330] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: stopping tablet replica
I20260811 22:33:31.279335 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 12750
I20260811 22:33:31.279350 13523 raft_consensus.cc:2229] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:31.279647 13520 raft_consensus.cc:2229] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:33:31.280085 13520 raft_consensus.cc:2258] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:31.280445 13522 tablet_replica.cc:330] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: stopping tablet replica
I20260811 22:33:31.280727 13522 raft_consensus.cc:2229] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:31.281008 13522 raft_consensus.cc:2258] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:31.291991 13520 ts_tablet_manager.cc:1905] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:31.294708 13522 ts_tablet_manager.cc:1905] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:31.294883 13520 ts_tablet_manager.cc:1918] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:33:31.294991 13520 log.cc:1196] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:31.295358 13520 ts_tablet_manager.cc:1939] T 51c1216a4edf44319a206ccf54a2227a P 221d520f4bc144d680a536275a4239d7: Deleting consensus metadata
I20260811 22:33:31.296664 12666 catalog_manager.cc:4841] TS 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189): tablet 51c1216a4edf44319a206ccf54a2227a (table test-table [id=8e2a64b8c34945b59c96757ae160763c]) successfully deleted
I20260811 22:33:31.298604 13522 ts_tablet_manager.cc:1918] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 22:33:31.298661 12665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:44781 (error 108)
I20260811 22:33:31.298822 13522 log.cc:1196] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:31.299068 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 13011
I20260811 22:33:31.303951 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.299499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44781 (local address 127.0.0.1:47604)
0811 22:33:31.303746 (+ 4247us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.303833 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:44781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4149}
W20260811 22:33:31.304154 12665 catalog_manager.cc:4584] TS 83205a60f3144ed3b072429678f08ff5 (127.18.211.129:44781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51c1216a4edf44319a206ccf54a2227a: Network error: Client connection negotiation failed: client connection to 127.18.211.129:44781: connect: Connection refused (error 111)
I20260811 22:33:31.306845 13522 ts_tablet_manager.cc:1939] T 51c1216a4edf44319a206ccf54a2227a P a3106d265a594a869de88bb55cd2391a: Deleting consensus metadata
I20260811 22:33:31.308228 12664 catalog_manager.cc:4841] TS a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393): tablet 51c1216a4edf44319a206ccf54a2227a (table test-table [id=8e2a64b8c34945b59c96757ae160763c]) successfully deleted
I20260811 22:33:31.314566 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 13242
I20260811 22:33:31.332219 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 12625
I20260811 22:33:31.346400 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44989
--webserver_interface=127.18.211.190
--webserver_port=39345
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:44989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:31.459422 13525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:31.459669 13525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:31.459708 13525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:31.463625 13525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:31.463678 13525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:31.463697 13525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:31.463742 13525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:31.463778 13525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:31.468786 13525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:44989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:44989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=39345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:31.469825 13525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:31.470857 13525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:31.479544 13534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:31.479614 13525 server_base.cc:1034] running on GCE node
W20260811 22:33:31.479553 13536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:31.479552 13538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:31.480417 13525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:31.481511 13525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:31.482685 13525 hybrid_clock.cc:648] HybridClock initialized: now 1786487611482671 us; error 28 us; skew 500 ppm
Aug 11 22:33:31 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:31.485507 13525 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:31.486379 13525 webserver.cc:460] Webserver started at http://127.18.211.190:39345/ using document root <none> and password file <none>
I20260811 22:33:31.486940 13525 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:31.487035 13525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:31.490867 13525 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 22:33:31.493155 13556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.494429 13525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:33:31.494575 13525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d02403a59d954c6bb82894ce851b2155"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "7ebedd3b2db3cdfc5da96369ce8da45f"
server_key_iv: "c40a164f44375289870e06f822b42d7d"
server_key_version: "encryptionkey@0"
I20260811 22:33:31.495131 13525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:31.520231 13525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:31.523037 13525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:31.523247 13525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:31.530151 13525 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:44989
I20260811 22:33:31.530448 13633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:44989 every 8 connection(s)
I20260811 22:33:31.531299 13525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:31.533658 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 13525
I20260811 22:33:31.534207 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:44781
--local_ip_for_outbound_sockets=127.18.211.129
--tserver_master_addrs=127.18.211.190:44989
--webserver_port=34943
--webserver_interface=127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:31.536427 13634 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:33:31.538138 13634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Bootstrap starting.
I20260811 22:33:31.555891 13634 log.cc:824] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:31.567371 13634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:33:31.567755 13634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155: Bootstrap complete.
I20260811 22:33:31.571811 13634 raft_consensus.cc:348] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:31.572227 13634 raft_consensus.cc:729] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02403a59d954c6bb82894ce851b2155, State: Initialized, Role: FOLLOWER
I20260811 22:33:31.572702 13634 consensus_queue.cc:260] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:31.572850 13634 raft_consensus.cc:388] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:33:31.572950 13634 raft_consensus.cc:482] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:33:31.573071 13634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:33:31.574874 13634 raft_consensus.cc:504] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:31.575233 13634 leader_election.cc:302] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d02403a59d954c6bb82894ce851b2155; no voters:
I20260811 22:33:31.575572 13634 leader_election.cc:288] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:33:31.575678 13640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:33:31.575913 13640 raft_consensus.cc:686] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [term 3 LEADER]: Becoming Leader. State: Replica: d02403a59d954c6bb82894ce851b2155, State: Running, Role: LEADER
I20260811 22:33:31.576207 13640 consensus_queue.cc:237] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } }
I20260811 22:33:31.576800 13634 sys_catalog.cc:564] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:31.577165 13640 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d02403a59d954c6bb82894ce851b2155. Latest consensus state: current_term: 3 leader_uuid: "d02403a59d954c6bb82894ce851b2155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } } }
I20260811 22:33:31.577303 13640 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:31.577690 13641 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d02403a59d954c6bb82894ce851b2155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02403a59d954c6bb82894ce851b2155" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 44989 } } }
I20260811 22:33:31.577885 13641 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02403a59d954c6bb82894ce851b2155 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:31.577764 13644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:31.583319 13644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:31.584090 13644 catalog_manager.cc:1259] Loaded cluster ID: a6edb518b12d45f5ba3aa3dee6103bae
I20260811 22:33:31.584156 13644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 22:33:31.585235 13644 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f20adf8a9ad google::LogMessage::Fail() at ??:0
@ 0x7f20adf8d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f20adf8a4de google::LogMessage::Flush() at ??:0
@ 0x7f20adf8ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f20b88c9fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f20b88cad2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f20b88c7579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f20b890b640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f20b01dc3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f20b01dcc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f20b01de3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f20b01cd512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f20b18116db start_thread at ??:0
@ 0x7f20aca8861f clone at ??:0
W20260811 22:33:31.675740 13638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:31.675997 13638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:31.676038 13638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:31.680022 13638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:31.680079 13638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:31.680135 13638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:33:31.685352 13638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:44781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=34943
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:31.686394 13638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:31.687417 13638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:31.697305 13702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:31.697300 13704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:31.697662 13707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:31.697655 13638 server_base.cc:1034] running on GCE node
I20260811 22:33:31.698120 13638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:31.698758 13638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:31.699990 13638 hybrid_clock.cc:648] HybridClock initialized: now 1786487611699939 us; error 48 us; skew 500 ppm
Aug 11 22:33:31 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:31.703061 13638 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:33:31.703979 13638 webserver.cc:460] Webserver started at http://127.18.211.129:34943/ using document root <none> and password file <none>
I20260811 22:33:31.704568 13638 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:31.704633 13638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:31.708679 13638 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:33:31.711102 13720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.712532 13638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:33:31.712682 13638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "83205a60f3144ed3b072429678f08ff5"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "196a9d0706b88c1a66c9bf977329268f"
server_key_iv: "d5acff0820ee6b9fa326a20d67368d34"
server_key_version: "encryptionkey@0"
I20260811 22:33:31.713274 13638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:31.742969 13638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:31.746428 13638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:31.746654 13638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:31.747321 13638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:31.748505 13732 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:33:31.754487 13638 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:33:31.754598 13638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 22:33:31.754712 13638 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:33:31.756372 13638 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:33:31.756446 13638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 22:33:31.756755 13732 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Bootstrap starting.
I20260811 22:33:31.767485 13638 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:44781
I20260811 22:33:31.767506 13913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:44781 every 8 connection(s)
I20260811 22:33:31.768455 13638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:33:31.772655 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 13638
I20260811 22:33:31.773066 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:43189
--local_ip_for_outbound_sockets=127.18.211.130
--tserver_master_addrs=127.18.211.190:44989
--webserver_port=33563
--webserver_interface=127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:31.779285 13732 log.cc:824] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:31.789352 13732 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:33:31.789746 13732 tablet_bootstrap.cc:492] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Bootstrap complete.
I20260811 22:33:31.791033 13732 ts_tablet_manager.cc:1397] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Time spent bootstrapping tablet: real 0.035s user 0.023s sys 0.008s
I20260811 22:33:31.793869 13732 raft_consensus.cc:348] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.794361 13732 raft_consensus.cc:729] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83205a60f3144ed3b072429678f08ff5, State: Initialized, Role: FOLLOWER
I20260811 22:33:31.794893 13732 consensus_queue.cc:260] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:31.795985 13732 ts_tablet_manager.cc:1428] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 22:33:31.797375 13916 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:33:31.770468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:59347)
0811 22:33:31.770935 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.770987 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:31.771831 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:31.796331 (+ 24500us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":151,"threads_started":1}
W20260811 22:33:31.797976 13914 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:44989 (0 consecutive failures): Network error: Failed to ping master at 127.18.211.190:44989: Client connection negotiation failed: client connection to 127.18.211.190:44989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:33:31.798748 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.798446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:57719)
0811 22:33:31.798605 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.798679 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:33:31.799547 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.799280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:53967)
0811 22:33:31.799398 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.799492 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:33:31.799762 13914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:33:31.896742 13917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:31.897011 13917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:31.897100 13917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:31.901175 13917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:31.901295 13917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:31.901396 13917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:33:31.906613 13917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:43189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=33563
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:31.907650 13917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:31.908672 13917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:31.919585 13925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:31.919598 13929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:31.919607 13924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:31.919962 13917 server_base.cc:1034] running on GCE node
I20260811 22:33:31.920400 13917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:31.921025 13917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:31.922223 13917 hybrid_clock.cc:648] HybridClock initialized: now 1786487611922206 us; error 56 us; skew 500 ppm
Aug 11 22:33:31 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:31.925076 13917 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:33:31.925944 13917 webserver.cc:460] Webserver started at http://127.18.211.130:33563/ using document root <none> and password file <none>
I20260811 22:33:31.926467 13917 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:31.926525 13917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:31.930274 13917 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:33:31.932315 13941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.933606 13917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:33:31.933730 13917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "221d520f4bc144d680a536275a4239d7"
format_stamp: "Formatted at 2026-08-11 22:33:29 on dist-test-slave-8hhm"
server_key: "b07e4031899dd6c9031ce6db7cfcf6c7"
server_key_iv: "4278be1ae00f5455a7f956ff5a7241a9"
server_key_version: "encryptionkey@0"
I20260811 22:33:31.934273 13917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:31.953473 13917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:31.956741 13917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:31.956943 13917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:31.957582 13917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:31.958413 13917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:31.958459 13917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.958523 13917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:31.958555 13917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:31.967674 13917 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:43189
I20260811 22:33:31.967705 14121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:43189 every 8 connection(s)
I20260811 22:33:31.968670 13917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:33:31.972018 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 13917
I20260811 22:33:31.972453 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:44393
--local_ip_for_outbound_sockets=127.18.211.131
--tserver_master_addrs=127.18.211.190:44989
--webserver_port=46769
--webserver_interface=127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:33:31.972811 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.970831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:42659)
0811 22:33:31.971302 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.971414 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":133,"threads_started":1}
W20260811 22:33:31.973330 14122 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:44989 (0 consecutive failures): Network error: Failed to ping master at 127.18.211.190:44989: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
I20260811 22:33:31.973976 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.973767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:37985)
0811 22:33:31.973876 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.973923 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:33:31.974732 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:31.974439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:43623)
0811 22:33:31.974597 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:31.974654 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:33:31.974938 14122 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:33:32.086640 14126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:32.086880 14126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:32.086921 14126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:32.090916 14126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:32.090967 14126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:32.091040 14126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:33:32.095995 14126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:44393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=46769
--tserver_master_addrs=127.18.211.190:44989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:32.097034 14126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:32.098063 14126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:33:32.107751 14126 server_base.cc:1034] running on GCE node
W20260811 22:33:32.107715 14140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:32.107774 14135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:32.107875 14134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:32.108480 14126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:32.109136 14126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:32.110374 14126 hybrid_clock.cc:648] HybridClock initialized: now 1786487612110333 us; error 50 us; skew 500 ppm
Aug 11 22:33:32 dist-test-slave-8hhm krb5kdc[11722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487612, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:32.113189 14126 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:33:32.114143 14126 webserver.cc:460] Webserver started at http://127.18.211.131:46769/ using document root <none> and password file <none>
I20260811 22:33:32.114713 14126 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:32.114810 14126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:32.118786 14126 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:33:32.120862 14153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:32.122123 14126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:33:32.122272 14126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "a3106d265a594a869de88bb55cd2391a"
format_stamp: "Formatted at 2026-08-11 22:33:30 on dist-test-slave-8hhm"
server_key: "b6bcad17abb7d7ed0d9e4d33c7229a16"
server_key_iv: "3e2f826c3d9b30183d3dff5ba792eee7"
server_key_version: "encryptionkey@0"
I20260811 22:33:32.122828 14126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:32.154081 14126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:32.156988 14126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:32.157236 14126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:32.157864 14126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:32.158717 14126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:32.158798 14126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:32.158876 14126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:32.158931 14126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:32.167824 14126 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:44393
I20260811 22:33:32.167855 14343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:44393 every 8 connection(s)
I20260811 22:33:32.168823 14126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:33:32.170467 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 14126
I20260811 22:33:32.173032 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:32.171099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:55649)
0811 22:33:32.171645 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:32.171793 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":151,"threads_started":1}
W20260811 22:33:32.173497 14344 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:44989 (0 consecutive failures): Network error: Failed to ping master at 127.18.211.190:44989: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
I20260811 22:33:32.174101 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:32.173879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:47113)
0811 22:33:32.173992 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:32.174039 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 22:33:32.174674 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:32.174478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:33909)
0811 22:33:32.174578 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:32.174620 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:33:32.174853 14344 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:33:32.800940 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:32.800478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:34299)
0811 22:33:32.800762 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:32.800823 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":99,"threads_started":1}
I20260811 22:33:32.976070 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:32.975591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:38853)
0811 22:33:32.975895 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:32.975962 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":97,"threads_started":1}
I20260811 22:33:33.175895 14356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:33.175492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:36151)
0811 22:33:33.175740 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:33.175797 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":74,"threads_started":1}
I20260811 22:33:33.352286 14358 raft_consensus.cc:482] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:33.352521 14358 raft_consensus.cc:504] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:33.353783 14358 leader_election.cc:288] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393), 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189)
Aug 11 22:33:33 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.131@KRBTEST.COM
Aug 11 22:33:33 dist-test-slave-8hhm krb5kdc[11722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.130@KRBTEST.COM
I20260811 22:33:33.367817 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:33.353940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:44393 (local address 127.18.211.129:41997)
0811 22:33:33.354307 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:33.354325 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:33.354438 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:33.355991 (+ 1553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:33.355997 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:33.356631 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:33.357120 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:33.357135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.358477 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.358480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:33.359377 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:33.359387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.360118 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.360122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:33.360338 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:33.361124 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:33.361151 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:33.362820 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:33.365037 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:33.365043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:33.365058 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:33.365454 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:33.365878 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:33.365880 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:33.365882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:33.366081 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:33.366796 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:33.366802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:33.367084 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 22:33:33.367286 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 22:33:33.367622 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":68,"spinlock_wait_cycles":4480,"thread_start_us":98,"threads_started":1}
I20260811 22:33:33.367831 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:33.354239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:52907 (local address 127.18.211.130:43189)
0811 22:33:33.354639 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:33.354659 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:33.354699 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:33:33.355332 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:33.355337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:33.355626 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:33.355940 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:33.357321 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.358312 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.359507 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.359968 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.360134 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:33.363380 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:33.363385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:33.363395 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:33.363420 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:33.365503 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:33.365994 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:33.366000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:33.366005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:33.366138 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:33.366427 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:33.366429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:33.366430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:33.366847 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:33.366986 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:33.367404 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 22:33:33.367664 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":94,"threads_started":1}
I20260811 22:33:33.367901 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:33.354128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:41997 (local address 127.18.211.131:44393)
0811 22:33:33.354448 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:33.354473 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:33.354527 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:33:33.355265 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:33.355271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:33.355612 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:33.355940 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:33.357363 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.358326 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.359522 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.359975 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.360152 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:33.363082 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:33.363090 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:33.363108 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:33.363143 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:33.364913 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:33.365587 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:33.365596 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:33.365603 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:33.365771 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:33.366209 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:33.366214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:33.366215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:33.366673 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:33.366776 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:33.367417 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 22:33:33.367702 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":79,"threads_started":1}
I20260811 22:33:33.368417 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:33.354071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:43189 (local address 127.18.211.129:52907)
0811 22:33:33.354474 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:33.354488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:33.354591 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:33.355992 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:33.355997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:33.356630 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:33.357121 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:33.357136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.358468 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.358473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:33.359377 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:33.359387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:33.360097 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:33.360101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:33.360346 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:33.361424 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:33.361437 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:33.363161 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:33.365618 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:33.365621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:33.365623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:33.365886 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:33.366240 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:33.366243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:33.366245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:33.366333 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:33.366969 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:33.366971 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:33.367075 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 22:33:33.367286 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:33:33.367714 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":348,"spinlock_wait_cycles":31104,"thread_start_us":70,"threads_started":1}
W20260811 22:33:33.370500 13727 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
W20260811 22:33:33.370666 13724 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:33.370751 13724 leader_election.cc:302] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83205a60f3144ed3b072429678f08ff5; no voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a
I20260811 22:33:33.370980 14358 raft_consensus.cc:2726] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:33:33.802091 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:33.801811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:57967)
0811 22:33:33.801946 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:33.802002 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 22:33:33.977324 14368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:33.976915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:33713)
0811 22:33:33.977159 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:33.977236 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260811 22:33:34.177040 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:34.176613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:56527)
0811 22:33:34.176897 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:34.176954 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":88,"threads_started":1}
I20260811 22:33:34.803452 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:34.803019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:56667)
0811 22:33:34.803291 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:34.803351 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":83,"threads_started":1}
I20260811 22:33:34.978564 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:34.978103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:44555)
0811 22:33:34.978398 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:34.978455 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":122,"threads_started":1}
I20260811 22:33:35.178244 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:35.177807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:54079)
0811 22:33:35.178064 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:35.178122 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
I20260811 22:33:35.217373 14374 raft_consensus.cc:482] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:35.217547 14374 raft_consensus.cc:504] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:35.218106 14374 leader_election.cc:288] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393), 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189)
W20260811 22:33:35.219008 13727 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
W20260811 22:33:35.219120 13724 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:35.219195 13724 leader_election.cc:302] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83205a60f3144ed3b072429678f08ff5; no voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a
I20260811 22:33:35.219338 14374 raft_consensus.cc:2726] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:33:35.804603 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:35.804201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:39861)
0811 22:33:35.804449 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:35.804504 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260811 22:33:35.979691 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:35.979282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:39695)
0811 22:33:35.979545 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:35.979597 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260811 22:33:36.179407 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:36.178994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:60571)
0811 22:33:36.179262 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:36.179317 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260811 22:33:36.806293 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:36.805828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:53813)
0811 22:33:36.806131 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:36.806191 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":98,"threads_started":1}
I20260811 22:33:36.981031 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:36.980603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:45915)
0811 22:33:36.980885 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:36.980940 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":108,"threads_started":1}
I20260811 22:33:37.180886 14386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:37.180435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:57365)
0811 22:33:37.180725 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:37.180787 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260811 22:33:37.720468 14387 raft_consensus.cc:482] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:37.720624 14387 raft_consensus.cc:504] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:37.721081 14387 leader_election.cc:288] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393), 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189)
W20260811 22:33:37.721946 13724 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
W20260811 22:33:37.722082 13727 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:37.722196 13727 leader_election.cc:302] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83205a60f3144ed3b072429678f08ff5; no voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a
I20260811 22:33:37.722383 14387 raft_consensus.cc:2726] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:33:37.807720 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:37.807143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:37307)
0811 22:33:37.807492 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:37.807552 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":128,"threads_started":1}
I20260811 22:33:37.982239 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:37.981821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:45069)
0811 22:33:37.982074 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:37.982132 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260811 22:33:38.182063 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:38.181644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:33187)
0811 22:33:38.181906 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:38.181962 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260811 22:33:38.809000 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:38.808557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:58945)
0811 22:33:38.808832 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:38.808894 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260811 22:33:38.983494 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:38.983059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:47617)
0811 22:33:38.983350 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:38.983404 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":71,"threads_started":1}
I20260811 22:33:39.183276 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:39.182819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:46749)
0811 22:33:39.183115 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:39.183167 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":75,"threads_started":1}
I20260811 22:33:39.810266 14398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:39.809843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:32859)
0811 22:33:39.810097 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:39.810155 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260811 22:33:39.984690 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:39.984304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:45765)
0811 22:33:39.984536 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:39.984589 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260811 22:33:40.184571 14400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:40.184164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:40411)
0811 22:33:40.184426 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:40.184481 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260811 22:33:40.439881 14401 raft_consensus.cc:482] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:40.440059 14401 raft_consensus.cc:504] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:40.440532 14401 leader_election.cc:288] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393), 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189)
W20260811 22:33:40.441339 13724 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
W20260811 22:33:40.441458 13727 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:40.441581 13727 leader_election.cc:302] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83205a60f3144ed3b072429678f08ff5; no voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a
I20260811 22:33:40.441756 14401 raft_consensus.cc:2726] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:33:40.811621 14402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:40.811201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:33917)
0811 22:33:40.811463 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:40.811521 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260811 22:33:40.986030 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:40.985585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:42223)
0811 22:33:40.985872 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:40.985935 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":79,"threads_started":1}
I20260811 22:33:41.185882 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:41.185479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:52285)
0811 22:33:41.185729 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:41.185790 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260811 22:33:41.813035 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:41.812555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:55445)
0811 22:33:41.812873 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:41.812934 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":98,"threads_started":1}
I20260811 22:33:41.987278 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:41.986850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:50525)
0811 22:33:41.987120 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:41.987177 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260811 22:33:42.187201 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:42.186729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:40471)
0811 22:33:42.187034 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:42.187098 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":79,"threads_started":1}
I20260811 22:33:42.814296 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:42.813868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:39597)
0811 22:33:42.814135 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:42.814192 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260811 22:33:42.988391 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:42.987989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:51287)
0811 22:33:42.988239 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:42.988293 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260811 22:33:43.188423 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:43.187972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:53461)
0811 22:33:43.188271 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:43.188327 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
I20260811 22:33:43.815656 14426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:43.815162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:57869)
0811 22:33:43.815441 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:43.815554 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":91,"threads_started":1}
I20260811 22:33:43.989591 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:43.989114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:32859)
0811 22:33:43.989437 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:43.989496 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":144,"threads_started":1}
I20260811 22:33:44.189792 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:44.189348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:59709)
0811 22:33:44.189631 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:44.189698 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":86,"threads_started":1}
I20260811 22:33:44.642180 14429 raft_consensus.cc:482] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:44.642316 14429 raft_consensus.cc:504] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3106d265a594a869de88bb55cd2391a" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 44393 } } peers { permanent_uuid: "221d520f4bc144d680a536275a4239d7" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 43189 } } peers { permanent_uuid: "83205a60f3144ed3b072429678f08ff5" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44781 } }
I20260811 22:33:44.642724 14429 leader_election.cc:288] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393), 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189)
W20260811 22:33:44.643472 13724 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3106d265a594a869de88bb55cd2391a (127.18.211.131:44393): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
W20260811 22:33:44.643604 13727 leader_election.cc:341] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 221d520f4bc144d680a536275a4239d7 (127.18.211.130:43189): Not found: Tablet not found: 51c1216a4edf44319a206ccf54a2227a
I20260811 22:33:44.643704 13727 leader_election.cc:302] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83205a60f3144ed3b072429678f08ff5; no voters: 221d520f4bc144d680a536275a4239d7, a3106d265a594a869de88bb55cd2391a
I20260811 22:33:44.643862 14429 raft_consensus.cc:2726] T 51c1216a4edf44319a206ccf54a2227a P 83205a60f3144ed3b072429678f08ff5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:33:44.816993 14430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:44.816528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:48921)
0811 22:33:44.816794 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:44.816860 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260811 22:33:44.990847 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:44.990438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:41809)
0811 22:33:44.990694 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:44.990749 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260811 22:33:45.190945 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:45.190519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:50981)
0811 22:33:45.190790 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:45.190847 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":80,"threads_started":1}
I20260811 22:33:45.818185 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:45.817774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:48071)
0811 22:33:45.818025 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:45.818084 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260811 22:33:45.991998 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:45.991559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:59349)
0811 22:33:45.991845 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:45.991899 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":96,"threads_started":1}
I20260811 22:33:46.192246 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:46.191830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:46771)
0811 22:33:46.192086 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:46.192144 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":81,"threads_started":1}
I20260811 22:33:46.819623 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:46.819173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.129:41335)
0811 22:33:46.819455 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:46.819521 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":78,"threads_started":1}
I20260811 22:33:46.993430 14441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:46.992936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.130:43215)
0811 22:33:46.993266 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:46.993330 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
I20260811 22:33:47.181517 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 13638
I20260811 22:33:47.194669 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:47.193398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:44989 (local address 127.18.211.131:39595)
0811 22:33:47.194402 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:47.194521 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":756,"threads_started":1}
I20260811 22:33:47.200700 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 13917
I20260811 22:33:47.216049 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 14126
2026-08-11T22:33:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19535 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:33:47 dist-test-slave-8hhm krb5kdc[14445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:33:47 dist-test-slave-8hhm krb5kdc[14445](info): set up 2 sockets
Aug 11 22:33:47 dist-test-slave-8hhm krb5kdc[14445](info): commencing operation
krb5kdc: starting...
W20260811 22:33:48.522418 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:33:48 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:33:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:33:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:33:48.656761 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45415
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:48.771631 14465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:48.771936 14465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:48.771977 14465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:48.776535 14465 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:33:48.776618 14465 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:33:48.776634 14465 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:33:48.776648 14465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:33:48.776665 14465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:48.776712 14465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:48.776748 14465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:33:48.776775 14465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:33:48.782068 14465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:48.783242 14465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:48.784358 14465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:33:48.793566 14465 server_base.cc:1034] running on GCE node
W20260811 22:33:48.793654 14476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:48.793654 14478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:48.793790 14480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:48.794513 14465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:48.795642 14465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:48.796856 14465 hybrid_clock.cc:648] HybridClock initialized: now 1786487628796836 us; error 31 us; skew 500 ppm
Aug 11 22:33:48 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487628, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:48.799748 14465 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:33:48.800726 14465 webserver.cc:460] Webserver started at http://127.18.211.190:44369/ using document root <none> and password file <none>
I20260811 22:33:48.801389 14465 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:48.801488 14465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:48.801751 14465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:48.804059 14465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "e2500f8bfe944e3e8880c653d577b433"
format_stamp: "Formatted at 2026-08-11 22:33:48 on dist-test-slave-8hhm"
server_key: "4d1ea61e1a72aeaa01202f641cc10230"
server_key_iv: "ef4f9b3249e42b91ec173395713df6ac"
server_key_version: "encryptionkey@0"
I20260811 22:33:48.804656 14465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "e2500f8bfe944e3e8880c653d577b433"
format_stamp: "Formatted at 2026-08-11 22:33:48 on dist-test-slave-8hhm"
server_key: "4d1ea61e1a72aeaa01202f641cc10230"
server_key_iv: "ef4f9b3249e42b91ec173395713df6ac"
server_key_version: "encryptionkey@0"
I20260811 22:33:48.808544 14465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:33:48.811064 14494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:48.812286 14465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:33:48.812438 14465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "e2500f8bfe944e3e8880c653d577b433"
format_stamp: "Formatted at 2026-08-11 22:33:48 on dist-test-slave-8hhm"
server_key: "4d1ea61e1a72aeaa01202f641cc10230"
server_key_iv: "ef4f9b3249e42b91ec173395713df6ac"
server_key_version: "encryptionkey@0"
I20260811 22:33:48.812554 14465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:48.829548 14465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:48.832716 14465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:48.832955 14465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:48.840833 14465 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45415
I20260811 22:33:48.841142 14590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45415 every 8 connection(s)
I20260811 22:33:48.841903 14465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:33:48.843995 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 14465
I20260811 22:33:48.844131 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:33:48.844432 19278 external_mini_cluster.cc:1428] Setting key 67348c34305884802b0a054e36eb281a
I20260811 22:33:48.845003 14593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:48.852283 14593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Bootstrap starting.
Aug 11 22:33:48 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:48.855012 14593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:48.855947 14593 log.cc:824] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:48.858073 14593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: No bootstrap required, opened a new log
I20260811 22:33:48.859930 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:48.846042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.18.211.190:45415)
0811 22:33:48.846467 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:48.846486 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:48.846533 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:33:48.847208 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:48.847226 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:48.847564 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:48.847849 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:48.848798 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:48.849894 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:48.851089 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:48.851565 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:48.851769 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:48.854696 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:48.854702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:48.854716 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:48.854746 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:48.856748 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:48.857277 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:48.857282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:48.857287 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:48.857405 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:48.857705 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:48.857712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:48.857716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:48.858164 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:48.858388 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:48.858693 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 22:33:48.859003 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":150,"threads_started":1}
I20260811 22:33:48.861402 14593 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:33:48.861702 14593 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:48.861763 14593 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2500f8bfe944e3e8880c653d577b433, State: Initialized, Role: FOLLOWER
I20260811 22:33:48.862236 14593 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:33:48.862401 14593 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:33:48.862473 14593 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:33:48.862547 14593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:48.863425 14593 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:33:48.863729 14593 leader_election.cc:302] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2500f8bfe944e3e8880c653d577b433; no voters:
I20260811 22:33:48.863989 14593 leader_election.cc:288] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:33:48.864128 14601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:48.864398 14601 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 LEADER]: Becoming Leader. State: Replica: e2500f8bfe944e3e8880c653d577b433, State: Running, Role: LEADER
I20260811 22:33:48.864763 14601 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:33:48.865325 14593 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:33:48.866760 14604 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2500f8bfe944e3e8880c653d577b433. Latest consensus state: current_term: 1 leader_uuid: "e2500f8bfe944e3e8880c653d577b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } } }
I20260811 22:33:48.866770 14603 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2500f8bfe944e3e8880c653d577b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } } }
I20260811 22:33:48.867030 14603 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:48.867030 14604 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: This master's current role is: LEADER
I20260811 22:33:48.867434 14616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:33:48.870705 14616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:33:48.876796 14616 catalog_manager.cc:1347] Generated new cluster ID: d1382cdf1afc494b9207716ab8e9597d
I20260811 22:33:48.876876 14616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:33:48.888345 14616 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:33:48.889761 14616 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:33:48.896126 14616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Generated new TSK 0
I20260811 22:33:48.896986 14616 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:33:48.932514 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45415
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:49.067632 14634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:49.067890 14634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:49.067934 14634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:49.072013 14634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:49.072070 14634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:49.072125 14634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:33:49.077157 14634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:49.078169 14634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:49.079141 14634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:49.089026 14642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:49.088997 14640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:49.089059 14634 server_base.cc:1034] running on GCE node
W20260811 22:33:49.089010 14645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:49.089757 14634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:49.090322 14634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:49.091485 14634 hybrid_clock.cc:648] HybridClock initialized: now 1786487629091467 us; error 29 us; skew 500 ppm
Aug 11 22:33:49 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487629, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:49.094203 14634 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:33:49.094987 14634 webserver.cc:460] Webserver started at http://127.18.211.129:42595/ using document root <none> and password file <none>
I20260811 22:33:49.095487 14634 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:49.095547 14634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:49.095718 14634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:49.097707 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "eb245fc55cb0481584e61bdf8204759a"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "5427122c4bd5faf73e4b4fe4c32e56a3"
server_key_iv: "1fea3377dfe0b3f9f9fbf4121f0fcb9b"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.098205 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "eb245fc55cb0481584e61bdf8204759a"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "5427122c4bd5faf73e4b4fe4c32e56a3"
server_key_iv: "1fea3377dfe0b3f9f9fbf4121f0fcb9b"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.101675 14634 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:33:49.103945 14656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.105154 14634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:33:49.105331 14634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "eb245fc55cb0481584e61bdf8204759a"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "5427122c4bd5faf73e4b4fe4c32e56a3"
server_key_iv: "1fea3377dfe0b3f9f9fbf4121f0fcb9b"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.105427 14634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:49.136150 14634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:49.139635 14634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:49.139806 14634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:49.140465 14634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:49.141352 14634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:49.141398 14634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.141469 14634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:49.141501 14634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.150316 14634 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:43255
I20260811 22:33:49.150410 14852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:43255 every 8 connection(s)
I20260811 22:33:49.151212 14634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
Aug 11 22:33:49 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487629, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:49.160593 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 14634
I20260811 22:33:49.160753 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:33:49.161072 19278 external_mini_cluster.cc:1428] Setting key 7e0d380661ffd0dd146165cee9047c89
I20260811 22:33:49.164657 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.153016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:54381 (local address 127.18.211.190:45415)
0811 22:33:49.153167 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.153172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.154269 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 22:33:49.155076 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.155078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.155123 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.155199 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:49.156360 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.156890 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.158064 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.158290 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.158372 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.160727 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:49.160731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:49.160733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:49.160758 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:49.162575 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.163122 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.163126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.163128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.163217 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.163566 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.163570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.163572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.163763 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:49.163876 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.164354 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.164483 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:33:49.165584 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.153475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:54381)
0811 22:33:49.154111 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.154153 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.154898 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.155384 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.155390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.155750 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:49.156223 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.156236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.157001 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.157003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.157956 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.157966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.158426 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.158428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.158565 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.159098 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:49.159115 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:49.160545 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:49.162694 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:49.162700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:49.162714 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:49.163009 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:49.163315 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:49.163318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:49.163320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:49.163464 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:49.163876 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:49.163884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:49.164114 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.164309 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.164615 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":116,"threads_started":1}
I20260811 22:33:49.166841 14854 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45415
I20260811 22:33:49.167088 14854 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:49.167629 14854 heartbeater.cc:511] Master 127.18.211.190:45415 requested a full tablet report, sending...
I20260811 22:33:49.169404 14526 ts_manager.cc:194] Registered new tserver with Master: eb245fc55cb0481584e61bdf8204759a (127.18.211.129:43255)
I20260811 22:33:49.171123 14526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:54381
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:33:49.191664 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45415
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:49.303371 14867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:49.303617 14867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:49.303658 14867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:49.307477 14867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:49.307530 14867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:49.307583 14867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:33:49.312676 14867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:49.313606 14867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:49.314543 14867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:33:49.324221 14874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:49.324231 14879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:49.324236 14876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:49.324460 14867 server_base.cc:1034] running on GCE node
I20260811 22:33:49.325089 14867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:49.325753 14867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:49.326959 14867 hybrid_clock.cc:648] HybridClock initialized: now 1786487629326948 us; error 31 us; skew 500 ppm
Aug 11 22:33:49 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487629, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:49.329823 14867 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:33:49.330751 14867 webserver.cc:460] Webserver started at http://127.18.211.130:45735/ using document root <none> and password file <none>
I20260811 22:33:49.331336 14867 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:49.331398 14867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:49.331646 14867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:49.333823 14867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "a9eb3ae72a364d478d5c449e22cedcb8"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "2ad665f20c423b6aefb7c0a2bf67ae52"
server_key_iv: "df070acae57d967cd7e6faf4386e9043"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.334393 14867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "a9eb3ae72a364d478d5c449e22cedcb8"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "2ad665f20c423b6aefb7c0a2bf67ae52"
server_key_iv: "df070acae57d967cd7e6faf4386e9043"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.338258 14867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:33:49.340997 14893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.342231 14867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:33:49.342358 14867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "a9eb3ae72a364d478d5c449e22cedcb8"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "2ad665f20c423b6aefb7c0a2bf67ae52"
server_key_iv: "df070acae57d967cd7e6faf4386e9043"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.342489 14867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:49.352891 14867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:49.355631 14867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:49.355832 14867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:49.356410 14867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:49.357333 14867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:49.357383 14867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.357448 14867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:49.357486 14867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.366348 14867 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:41071
I20260811 22:33:49.366379 15088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:41071 every 8 connection(s)
I20260811 22:33:49.367380 14867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:33:49.368400 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 14867
I20260811 22:33:49.368510 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:33:49.368872 19278 external_mini_cluster.cc:1428] Setting key 00fc4fd826681140c59dea88954d8478
Aug 11 22:33:49 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487629, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:33:49.382447 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.369438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:49441 (local address 127.18.211.190:45415)
0811 22:33:49.369553 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.369557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.370631 (+ 1074us) server_negotiation.cc:407] Connection header received
0811 22:33:49.371559 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.371565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.371628 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.371736 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:49.373281 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.373773 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.375518 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.375726 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.375800 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.378706 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:49.378710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:49.378713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:49.378743 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:49.380382 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.380937 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.380940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.380941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.381032 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.381405 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.381407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.381409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.381637 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:49.381750 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.382231 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.382329 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 22:33:49.383292 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.369771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:49441)
0811 22:33:49.370468 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.370519 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.371326 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.371921 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.371928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.372334 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:49.373077 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.373099 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.373906 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.373913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.375388 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.375403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.375893 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.375900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.376143 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.376747 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:49.376774 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:49.378521 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:49.380503 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:49.380507 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:49.380518 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:49.380831 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:49.381138 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:49.381140 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:49.381141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:49.381339 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:49.381765 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:49.381775 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:49.382013 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.382161 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.382403 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:33:49.384560 15089 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45415
I20260811 22:33:49.384840 15089 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:49.385474 15089 heartbeater.cc:511] Master 127.18.211.190:45415 requested a full tablet report, sending...
I20260811 22:33:49.386699 14526 ts_manager.cc:194] Registered new tserver with Master: a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071)
I20260811 22:33:49.387305 14526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:49441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:33:49.401116 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45415
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:33:49.516628 15098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:33:49.516884 15098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:33:49.517000 15098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:33:49.521031 15098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:33:49.521092 15098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:33:49.521152 15098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:33:49.526324 15098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:33:49.527374 15098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:33:49.528431 15098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:33:49.538528 15098 server_base.cc:1034] running on GCE node
W20260811 22:33:49.538472 15109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:49.538472 15110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:33:49.538472 15115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:33:49.539209 15098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:33:49.539913 15098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:33:49.541105 15098 hybrid_clock.cc:648] HybridClock initialized: now 1786487629541080 us; error 44 us; skew 500 ppm
Aug 11 22:33:49 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487629, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:33:49.543874 15098 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:33:49.544847 15098 webserver.cc:460] Webserver started at http://127.18.211.131:34565/ using document root <none> and password file <none>
I20260811 22:33:49.545536 15098 fs_manager.cc:359] Metadata directory not provided
I20260811 22:33:49.545631 15098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:33:49.545892 15098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:33:49.548017 15098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "4246ee5c3e94441580802a72bb3bec71"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "6b3846e4e290d10887c346dcb812bee8"
server_key_iv: "6ee7474715e0d8b7b572704cdda445f6"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.548588 15098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "4246ee5c3e94441580802a72bb3bec71"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "6b3846e4e290d10887c346dcb812bee8"
server_key_iv: "6ee7474715e0d8b7b572704cdda445f6"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.552331 15098 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 22:33:49.554833 15127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.556046 15098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:33:49.556188 15098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "4246ee5c3e94441580802a72bb3bec71"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "6b3846e4e290d10887c346dcb812bee8"
server_key_iv: "6ee7474715e0d8b7b572704cdda445f6"
server_key_version: "encryptionkey@0"
I20260811 22:33:49.556308 15098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:33:49.579068 15098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:33:49.582119 15098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:33:49.582350 15098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:33:49.582955 15098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:33:49.583814 15098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:33:49.583887 15098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.583953 15098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:33:49.584003 15098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:33:49.593427 15098 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:46783
I20260811 22:33:49.593456 15323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:46783 every 8 connection(s)
I20260811 22:33:49.594298 15098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:33:49.598253 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 15098
I20260811 22:33:49.598385 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:33:49.598730 19278 external_mini_cluster.cc:1428] Setting key 41126ccec8bafb22ade96cf6923894c2
Aug 11 22:33:49 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487629, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:33:49.607402 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.596071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:51345 (local address 127.18.211.190:45415)
0811 22:33:49.596224 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.596227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.597112 (+ 885us) server_negotiation.cc:407] Connection header received
0811 22:33:49.598027 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.598030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.598078 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.598147 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:49.599512 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.600270 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.601522 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.601717 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.601792 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.604139 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:49.604142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:49.604144 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:49.604169 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:49.605696 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.606141 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.606143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.606144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.606229 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.606515 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.606517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.606518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.606682 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:49.606756 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.607189 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.607287 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:33:49.608165 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.596363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:51345)
0811 22:33:49.596985 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.597024 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.597833 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.598333 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.598343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.598845 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:33:49.599379 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.599392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.600390 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.600393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.601420 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.601429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.601837 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.601840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.601963 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.602516 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:33:49.602532 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:33:49.603990 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:33:49.605794 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:49.605798 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:49.605809 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:49.606052 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:49.606307 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:33:49.606309 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:33:49.606311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:33:49.606436 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:33:49.606767 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:33:49.606771 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:33:49.606981 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.607128 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.607358 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":115,"threads_started":1}
I20260811 22:33:49.609452 15325 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45415
I20260811 22:33:49.609710 15325 heartbeater.cc:464] Registering TS with master...
I20260811 22:33:49.610224 15325 heartbeater.cc:511] Master 127.18.211.190:45415 requested a full tablet report, sending...
I20260811 22:33:49.611299 14526 ts_manager.cc:194] Registered new tserver with Master: 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
I20260811 22:33:49.611969 14526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:51345
I20260811 22:33:49.612600 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:33:49.621723 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.614846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49056 (local address 127.18.211.190:45415)
0811 22:33:49.614959 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.614962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.615147 (+ 185us) server_negotiation.cc:407] Connection header received
0811 22:33:49.615235 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.615237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.615270 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.615337 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:33:49.616081 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.616551 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.617415 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.617617 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.617698 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.618976 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:33:49.618978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:33:49.618980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:33:49.619000 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:33:49.620202 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.620572 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.620574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.620575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.620647 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:33:49.620875 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:33:49.620877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:33:49.620878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:33:49.621067 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:33:49.621170 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.621490 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.621614 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:33:49.624461 14525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:33:49.627456 14525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:33:49.641901 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.636021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:46783 (local address 127.0.0.1:41334)
0811 22:33:49.637110 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.637125 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.637218 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.637822 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.637825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.637844 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:49.638023 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.638032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.639354 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.639357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.640376 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.640386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.641318 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.641322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.641428 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.641472 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.641598 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.641729 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"mutex_wait_us":80,"thread_start_us":96,"threads_started":1}
I20260811 22:33:49.642128 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.635765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.18.211.130:41071)
0811 22:33:49.636590 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.636603 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.636629 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:33:49.637087 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.637094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.637333 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.637487 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:49.637834 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.639022 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.640652 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.641449 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.641609 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.641729 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.641858 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.641964 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":92,"threads_started":1}
I20260811 22:33:49.642409 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.636427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.18.211.131:46783)
0811 22:33:49.636894 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.636908 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.637262 (+ 354us) server_negotiation.cc:407] Connection header received
0811 22:33:49.637340 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.637346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.637521 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.637663 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:49.638165 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.639216 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.640505 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.641154 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.641332 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.641458 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.642119 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.642226 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":168,"threads_started":1}
I20260811 22:33:49.642643 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.635719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:43255 (local address 127.0.0.1:60650)
0811 22:33:49.637004 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.637020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.637116 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.637799 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.637803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.637825 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:49.637981 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.637986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.638953 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.638959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.640852 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.640863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.642217 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.642223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.642337 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.642360 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.642423 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.642508 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"mutex_wait_us":34,"thread_start_us":108,"threads_started":1}
I20260811 22:33:49.642078 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.635630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41071 (local address 127.0.0.1:57116)
0811 22:33:49.636223 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.636271 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.636841 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.637496 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.637502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.637524 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:49.637701 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.637710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.639171 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.639175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.640508 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.640520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.641589 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.641593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.641712 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.641738 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.641821 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.641891 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":40,"thread_start_us":74,"threads_started":1}
I20260811 22:33:49.643224 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.636416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.18.211.129:43255)
0811 22:33:49.636850 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.636866 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.637104 (+ 238us) server_negotiation.cc:407] Connection header received
0811 22:33:49.637261 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.637269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.637476 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.637633 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:49.638097 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.638818 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.640987 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.641872 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.641987 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.642908 (+ 921us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.642961 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.643047 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":117,"threads_started":1}
I20260811 22:33:49.645783 14739 tablet_service.cc:1467] Processing CreateTablet for tablet 21372da0158e45beae96db660f2505fd (DEFAULT_TABLE table=test-table [id=9a3801c2f89d4d149ee8d83c8bf8f7be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:49.645776 15207 tablet_service.cc:1467] Processing CreateTablet for tablet 21372da0158e45beae96db660f2505fd (DEFAULT_TABLE table=test-table [id=9a3801c2f89d4d149ee8d83c8bf8f7be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:49.645777 14984 tablet_service.cc:1467] Processing CreateTablet for tablet 21372da0158e45beae96db660f2505fd (DEFAULT_TABLE table=test-table [id=9a3801c2f89d4d149ee8d83c8bf8f7be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:33:49.646848 14984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21372da0158e45beae96db660f2505fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:49.646850 15207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21372da0158e45beae96db660f2505fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:49.646849 14739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21372da0158e45beae96db660f2505fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:33:49.653059 15351 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Bootstrap starting.
I20260811 22:33:49.654690 15353 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Bootstrap starting.
I20260811 22:33:49.655850 15351 tablet_bootstrap.cc:654] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:49.656796 15353 tablet_bootstrap.cc:654] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:49.657011 15351 log.cc:824] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:49.657286 15352 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Bootstrap starting.
I20260811 22:33:49.657696 15353 log.cc:824] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:49.659018 15351 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: No bootstrap required, opened a new log
I20260811 22:33:49.659235 15351 ts_tablet_manager.cc:1397] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:33:49.659775 15353 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: No bootstrap required, opened a new log
I20260811 22:33:49.659998 15353 ts_tablet_manager.cc:1397] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:33:49.660214 15352 tablet_bootstrap.cc:654] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Neither blocks nor log segments found. Creating new log.
I20260811 22:33:49.661103 15352 log.cc:824] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Log is configured to *not* fsync() on all Append() calls
I20260811 22:33:49.662810 15351 raft_consensus.cc:348] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.663057 15352 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: No bootstrap required, opened a new log
I20260811 22:33:49.663072 15351 raft_consensus.cc:374] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:49.663125 15351 raft_consensus.cc:729] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb245fc55cb0481584e61bdf8204759a, State: Initialized, Role: FOLLOWER
I20260811 22:33:49.663290 15352 ts_tablet_manager.cc:1397] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:33:49.663602 15351 consensus_queue.cc:260] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.663831 15353 raft_consensus.cc:348] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.664156 15353 raft_consensus.cc:374] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:49.664253 15353 raft_consensus.cc:729] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9eb3ae72a364d478d5c449e22cedcb8, State: Initialized, Role: FOLLOWER
I20260811 22:33:49.664283 14854 heartbeater.cc:503] Master 127.18.211.190:45415 was elected leader, sending a full tablet report...
I20260811 22:33:49.664640 15351 ts_tablet_manager.cc:1428] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:33:49.664757 15353 consensus_queue.cc:260] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.665621 15089 heartbeater.cc:503] Master 127.18.211.190:45415 was elected leader, sending a full tablet report...
I20260811 22:33:49.665892 15353 ts_tablet_manager.cc:1428] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:33:49.668227 15352 raft_consensus.cc:348] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.668519 15352 raft_consensus.cc:374] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:33:49.668588 15352 raft_consensus.cc:729] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4246ee5c3e94441580802a72bb3bec71, State: Initialized, Role: FOLLOWER
I20260811 22:33:49.669042 15352 consensus_queue.cc:260] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.669775 15325 heartbeater.cc:503] Master 127.18.211.190:45415 was elected leader, sending a full tablet report...
I20260811 22:33:49.670051 15352 ts_tablet_manager.cc:1428] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260811 22:33:49.734357 15357 raft_consensus.cc:482] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:33:49.734643 15357 raft_consensus.cc:504] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.735826 15357 leader_election.cc:288] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
I20260811 22:33:49.739822 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.736284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:52613 (local address 127.18.211.130:41071)
0811 22:33:49.736411 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.736415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.736552 (+ 137us) server_negotiation.cc:407] Connection header received
0811 22:33:49.736724 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.736730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.736779 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.736865 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:49.737135 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.737706 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.738830 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.739225 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.739352 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.739407 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.739617 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.739691 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:33:49.739842 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.736071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41071 (local address 127.18.211.129:52613)
0811 22:33:49.736455 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.736473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.736599 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.736879 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.736882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.736899 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:49.737026 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.737032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.737822 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.737825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.738692 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.738702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.739349 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.739352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.739449 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.739511 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.739599 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.739667 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":63,"thread_start_us":118,"threads_started":1}
I20260811 22:33:49.739845 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.736189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:50831 (local address 127.18.211.131:46783)
0811 22:33:49.736331 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.736336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.736622 (+ 286us) server_negotiation.cc:407] Connection header received
0811 22:33:49.736789 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.736792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.736844 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.736931 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:33:49.737263 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.737748 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.738826 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.739235 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.739365 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.739420 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.739655 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.739722 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:33:49.739867 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.736071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:46783 (local address 127.18.211.129:50831)
0811 22:33:49.736515 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.736527 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:33:49.736669 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:33:49.736944 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:33:49.736947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:33:49.736964 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:33:49.737107 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.737112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.737869 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.737873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.738684 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:33:49.738703 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.739363 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.739366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:33:49.739529 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.739547 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:33:49.739637 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:33:49.739716 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":100,"thread_start_us":127,"threads_started":1}
I20260811 22:33:49.740602 15251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21372da0158e45beae96db660f2505fd" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4246ee5c3e94441580802a72bb3bec71" is_pre_election: true
I20260811 22:33:49.740604 15014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21372da0158e45beae96db660f2505fd" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" is_pre_election: true
I20260811 22:33:49.740919 15014 raft_consensus.cc:2454] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 0.
I20260811 22:33:49.740921 15251 raft_consensus.cc:2454] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 0.
I20260811 22:33:49.741425 14659 leader_election.cc:302] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4246ee5c3e94441580802a72bb3bec71, a9eb3ae72a364d478d5c449e22cedcb8, eb245fc55cb0481584e61bdf8204759a; no voters:
I20260811 22:33:49.741715 15357 raft_consensus.cc:2781] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:33:49.741792 15357 raft_consensus.cc:482] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:33:49.741822 15357 raft_consensus.cc:3037] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:49.742888 15357 raft_consensus.cc:504] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.743265 15357 leader_election.cc:288] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 election: Requested vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
I20260811 22:33:49.743618 15014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21372da0158e45beae96db660f2505fd" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9eb3ae72a364d478d5c449e22cedcb8"
I20260811 22:33:49.743631 15251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21372da0158e45beae96db660f2505fd" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4246ee5c3e94441580802a72bb3bec71"
I20260811 22:33:49.743770 15014 raft_consensus.cc:3037] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:49.743769 15251 raft_consensus.cc:3037] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:33:49.745076 15251 raft_consensus.cc:2454] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 1.
I20260811 22:33:49.745076 15014 raft_consensus.cc:2454] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 1.
I20260811 22:33:49.745409 14659 leader_election.cc:302] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4246ee5c3e94441580802a72bb3bec71, a9eb3ae72a364d478d5c449e22cedcb8, eb245fc55cb0481584e61bdf8204759a; no voters:
I20260811 22:33:49.745631 15357 raft_consensus.cc:2781] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:33:49.745813 15357 raft_consensus.cc:686] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 LEADER]: Becoming Leader. State: Replica: eb245fc55cb0481584e61bdf8204759a, State: Running, Role: LEADER
I20260811 22:33:49.746160 15357 consensus_queue.cc:237] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:33:49.748983 14525 catalog_manager.cc:5495] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a reported cstate change: term changed from 0 to 1, leader changed from <none> to eb245fc55cb0481584e61bdf8204759a (127.18.211.129). New cstate: current_term: 1 leader_uuid: "eb245fc55cb0481584e61bdf8204759a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } health_report { overall_health: UNKNOWN } } }
I20260811 22:33:49.766000 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.762657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.18.211.129:43255)
0811 22:33:49.762796 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:33:49.762800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:33:49.762813 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:33:49.762962 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:33:49.762964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:33:49.763020 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:33:49.763109 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:33:49.763320 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.763820 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.764617 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:33:49.764828 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:33:49.764904 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:33:49.765162 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:33:49.765682 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:33:49.765785 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:33:49.765846 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 22:33:49.765895 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:33:49.772246 15014 raft_consensus.cc:1261] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Refusing update from remote peer eb245fc55cb0481584e61bdf8204759a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:49.772246 15251 raft_consensus.cc:1261] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Refusing update from remote peer eb245fc55cb0481584e61bdf8204759a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:33:49.772957 15357 consensus_queue.cc:1035] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:49.773187 15367 consensus_queue.cc:1035] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:33:49.781993 15372 mvcc.cc:205] Tried to move back new op lower bound from 7317453331539083264 to 7317453331442864128. Current Snapshot: MvccSnapshot[applied={T|T < 7317453331539083264}]
I20260811 22:33:49.791785 14526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:49.791973 14526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:33:49.794499 14526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Sending DeleteTablet for 3 replicas of tablet 21372da0158e45beae96db660f2505fd
I20260811 22:33:49.795269 14739 tablet_service.cc:1514] Processing DeleteTablet for tablet 21372da0158e45beae96db660f2505fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:60650
I20260811 22:33:49.795296 14984 tablet_service.cc:1514] Processing DeleteTablet for tablet 21372da0158e45beae96db660f2505fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:57116
I20260811 22:33:49.795830 15391 tablet_replica.cc:330] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: stopping tablet replica
I20260811 22:33:49.795849 15207 tablet_service.cc:1514] Processing DeleteTablet for tablet 21372da0158e45beae96db660f2505fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:41334
I20260811 22:33:49.796087 15392 tablet_replica.cc:330] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: stopping tablet replica
I20260811 22:33:49.796207 15391 raft_consensus.cc:2229] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:33:49.796386 15392 raft_consensus.cc:2229] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:49.796403 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 14634
I20260811 22:33:49.796764 15392 raft_consensus.cc:2258] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:49.808104 15396 tablet_replica.cc:330] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: stopping tablet replica
I20260811 22:33:49.808630 15396 raft_consensus.cc:2229] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:33:49.809168 15396 raft_consensus.cc:2258] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:33:49.810186 15392 ts_tablet_manager.cc:1905] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:49.810792 15396 ts_tablet_manager.cc:1905] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:33:49.813364 15392 ts_tablet_manager.cc:1918] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:33:49.813515 15392 log.cc:1196] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/21372da0158e45beae96db660f2505fd
I20260811 22:33:49.813668 15396 ts_tablet_manager.cc:1918] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:33:49.813787 15396 log.cc:1196] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/21372da0158e45beae96db660f2505fd
I20260811 22:33:49.813915 15392 ts_tablet_manager.cc:1939] T 21372da0158e45beae96db660f2505fd P a9eb3ae72a364d478d5c449e22cedcb8: Deleting consensus metadata
I20260811 22:33:49.814172 15396 ts_tablet_manager.cc:1939] T 21372da0158e45beae96db660f2505fd P 4246ee5c3e94441580802a72bb3bec71: Deleting consensus metadata
I20260811 22:33:49.815413 14499 catalog_manager.cc:4841] TS a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071): tablet 21372da0158e45beae96db660f2505fd (table test-table [id=9a3801c2f89d4d149ee8d83c8bf8f7be]) successfully deleted
I20260811 22:33:49.815413 14500 catalog_manager.cc:4841] TS 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783): tablet 21372da0158e45beae96db660f2505fd (table test-table [id=9a3801c2f89d4d149ee8d83c8bf8f7be]) successfully deleted
W20260811 22:33:49.816344 14495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:43255 (error 108)
I20260811 22:33:49.816653 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 14867
I20260811 22:33:49.827521 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:33:49.826995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:43255 (local address 127.0.0.1:60660)
0811 22:33:49.827294 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:33:49.827401 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:43255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260811 22:33:49.827768 14495 catalog_manager.cc:4584] TS eb245fc55cb0481584e61bdf8204759a (127.18.211.129:43255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21372da0158e45beae96db660f2505fd: Network error: Client connection negotiation failed: client connection to 127.18.211.129:43255: connect: Connection refused (error 111)
I20260811 22:33:49.833135 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 15098
I20260811 22:33:49.855154 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 14465
I20260811 22:34:09.866950 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45415
--webserver_interface=127.18.211.190
--webserver_port=44369
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:09.985133 15398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:09.985458 15398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:09.985502 15398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:09.989475 15398 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:34:09.989527 15398 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:34:09.989543 15398 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:34:09.989557 15398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:09.989575 15398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:09.989620 15398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:09.989660 15398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:09.989684 15398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:09.995023 15398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=44369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:09.996115 15398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:09.997164 15398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:10.012014 15409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:10.012022 15408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:10.012135 15398 server_base.cc:1034] running on GCE node
W20260811 22:34:10.011994 15411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:10.012914 15398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:10.014256 15398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:10.015455 15398 hybrid_clock.cc:648] HybridClock initialized: now 1786487650015422 us; error 42 us; skew 500 ppm
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:10.024947 15398 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:34:10.025939 15398 webserver.cc:460] Webserver started at http://127.18.211.190:44369/ using document root <none> and password file <none>
I20260811 22:34:10.026491 15398 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:10.026564 15398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:10.030539 15398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:34:10.032686 15427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.033933 15398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:10.034085 15398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "e2500f8bfe944e3e8880c653d577b433"
format_stamp: "Formatted at 2026-08-11 22:33:48 on dist-test-slave-8hhm"
server_key: "4d1ea61e1a72aeaa01202f641cc10230"
server_key_iv: "ef4f9b3249e42b91ec173395713df6ac"
server_key_version: "encryptionkey@0"
I20260811 22:34:10.034718 15398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:10.059031 15398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:10.062309 15398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:10.062529 15398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:10.069867 15398 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45415
I20260811 22:34:10.070209 15507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45415 every 8 connection(s)
I20260811 22:34:10.071053 15398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:10.075626 15512 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:34:10.076596 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 15398
I20260811 22:34:10.077023 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:43255
--local_ip_for_outbound_sockets=127.18.211.129
--tserver_master_addrs=127.18.211.190:45415
--webserver_port=42595
--webserver_interface=127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:34:10.077555 15512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Bootstrap starting.
I20260811 22:34:10.101989 15512 log.cc:824] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:10.113619 15512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:34:10.114029 15512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Bootstrap complete.
I20260811 22:34:10.118366 15512 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:10.118817 15512 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2500f8bfe944e3e8880c653d577b433, State: Initialized, Role: FOLLOWER
I20260811 22:34:10.119386 15512 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:10.119541 15512 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:10.119649 15512 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:10.119776 15512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:34:10.121783 15512 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:10.122138 15512 leader_election.cc:302] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2500f8bfe944e3e8880c653d577b433; no voters:
I20260811 22:34:10.122504 15512 leader_election.cc:288] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:34:10.122602 15516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:34:10.122817 15516 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 LEADER]: Becoming Leader. State: Replica: e2500f8bfe944e3e8880c653d577b433, State: Running, Role: LEADER
I20260811 22:34:10.123189 15516 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:10.123795 15512 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:10.124616 15517 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e2500f8bfe944e3e8880c653d577b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } } }
I20260811 22:34:10.124761 15517 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:10.125079 15518 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2500f8bfe944e3e8880c653d577b433. Latest consensus state: current_term: 2 leader_uuid: "e2500f8bfe944e3e8880c653d577b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } } }
I20260811 22:34:10.125293 15518 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:10.125607 15534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:10.130429 15534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:10.131129 15534 catalog_manager.cc:1259] Loaded cluster ID: d1382cdf1afc494b9207716ab8e9597d
I20260811 22:34:10.131197 15534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:10.133685 15534 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:10.134387 15534 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Loaded TSK: 0
I20260811 22:34:10.153043 15534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Generated new TSK 1
I20260811 22:34:10.153438 15534 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:34:10.212118 15514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:10.212364 15514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:10.212404 15514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:10.216696 15514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:10.216754 15514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:10.216856 15514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:10.221961 15514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:43255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=42595
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:10.222911 15514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:10.223913 15514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:10.233397 15555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:10.233414 15551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:10.233474 15557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:10.234418 15514 server_base.cc:1034] running on GCE node
I20260811 22:34:10.234848 15514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:10.235509 15514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:10.236722 15514 hybrid_clock.cc:648] HybridClock initialized: now 1786487650236698 us; error 42 us; skew 500 ppm
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:10.239856 15514 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:34:10.240824 15514 webserver.cc:460] Webserver started at http://127.18.211.129:42595/ using document root <none> and password file <none>
I20260811 22:34:10.241453 15514 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:10.241570 15514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:10.245572 15514 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:34:10.247758 15570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.249042 15514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:10.249178 15514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "eb245fc55cb0481584e61bdf8204759a"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "5427122c4bd5faf73e4b4fe4c32e56a3"
server_key_iv: "1fea3377dfe0b3f9f9fbf4121f0fcb9b"
server_key_version: "encryptionkey@0"
I20260811 22:34:10.249750 15514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:10.270692 15514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:10.273609 15514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:10.273816 15514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:10.274430 15514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:10.275569 15579 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:34:10.279779 15514 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:34:10.279837 15514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260811 22:34:10.279912 15514 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:34:10.281591 15514 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:34:10.281637 15514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 22:34:10.281911 15579 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Bootstrap starting.
I20260811 22:34:10.292507 15514 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:43255
I20260811 22:34:10.292580 15758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:43255 every 8 connection(s)
I20260811 22:34:10.293581 15514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:10.295920 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 15514
I20260811 22:34:10.296428 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:41071
--local_ip_for_outbound_sockets=127.18.211.130
--tserver_master_addrs=127.18.211.190:45415
--webserver_port=45735
--webserver_interface=127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:34:10.300981 15579 log.cc:824] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:10.311215 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.295719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:44889 (local address 127.18.211.190:45415)
0811 22:34:10.296194 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.296215 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.296668 (+ 453us) server_negotiation.cc:407] Connection header received
0811 22:34:10.297976 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.297987 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.298329 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.298728 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:10.300207 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.301318 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.302699 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.303239 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.303407 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.306047 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:10.306052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:10.306065 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:10.306101 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:10.307883 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.308458 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.308463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.308468 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.308591 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.308962 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.308965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.308966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.309387 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:10.309536 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.310104 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.310380 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":108,"threads_started":1}
I20260811 22:34:10.311226 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.295559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:44889)
0811 22:34:10.296480 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.296536 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.297514 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.298852 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.298862 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.299368 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:10.300017 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.300039 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.301482 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.301486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.302560 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.302571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.303384 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.303387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.303591 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.304261 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:10.304279 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:10.305838 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:10.308021 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.308027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.308041 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.308315 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.308696 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.308700 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.308702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.308846 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.309507 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:10.309513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:10.309773 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.309961 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.310303 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":158,"threads_started":1}
I20260811 22:34:10.311329 15579 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:34:10.311837 15579 tablet_bootstrap.cc:492] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Bootstrap complete.
I20260811 22:34:10.313352 15579 ts_tablet_manager.cc:1397] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Time spent bootstrapping tablet: real 0.032s user 0.015s sys 0.016s
I20260811 22:34:10.314052 15761 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45415
I20260811 22:34:10.314288 15761 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:10.314826 15761 heartbeater.cc:511] Master 127.18.211.190:45415 requested a full tablet report, sending...
I20260811 22:34:10.316411 15579 raft_consensus.cc:348] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:34:10.316941 15579 raft_consensus.cc:729] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb245fc55cb0481584e61bdf8204759a, State: Initialized, Role: FOLLOWER
I20260811 22:34:10.317201 15454 ts_manager.cc:194] Registered new tserver with Master: eb245fc55cb0481584e61bdf8204759a (127.18.211.129:43255)
I20260811 22:34:10.317512 15579 consensus_queue.cc:260] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } }
I20260811 22:34:10.318528 15579 ts_tablet_manager.cc:1428] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:34:10.320705 15454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:44889
I20260811 22:34:10.323002 15761 heartbeater.cc:503] Master 127.18.211.190:45415 was elected leader, sending a full tablet report...
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.129@KRBTEST.COM
I20260811 22:34:10.332500 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.320244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:43255 (local address 127.0.0.1:48756)
0811 22:34:10.320661 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.320707 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.320839 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.321620 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.321626 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.323755 (+ 2129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:10.323893 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.323902 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.325339 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.325342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.325984 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.325993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.326734 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.326737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.326815 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.327403 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:10.327415 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:10.328819 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:10.330896 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.330904 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.330909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.331132 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.331442 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.331443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.331445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.331520 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.332020 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:10.332024 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:10.332125 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.332233 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.332374 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":125,"threads_started":1}
I20260811 22:34:10.332664 15779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.320336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48756 (local address 127.18.211.129:43255)
0811 22:34:10.321014 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.321026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.321049 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:34:10.321129 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.321134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.321344 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.321480 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:10.324027 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.325187 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.326117 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.326621 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.326728 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.328958 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:10.328968 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:10.328980 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:10.329006 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:10.330779 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.331242 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.331248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.331250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.331345 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.331603 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.331606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.331608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.331905 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:10.332030 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.332343 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.332507 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":88,"threads_started":1}
I20260811 22:34:10.333762 15651 tablet_service.cc:1514] Processing DeleteTablet for tablet 21372da0158e45beae96db660f2505fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:33:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:48756
I20260811 22:34:10.334589 15780 tablet_replica.cc:330] stopping tablet replica
I20260811 22:34:10.334882 15780 raft_consensus.cc:2229] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:10.335143 15780 raft_consensus.cc:2258] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:10.336098 15780 ts_tablet_manager.cc:1905] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:10.340067 15780 ts_tablet_manager.cc:1918] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:10.340161 15780 log.cc:1196] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal/wals/21372da0158e45beae96db660f2505fd
I20260811 22:34:10.340449 15780 ts_tablet_manager.cc:1939] T 21372da0158e45beae96db660f2505fd P eb245fc55cb0481584e61bdf8204759a: Deleting consensus metadata
I20260811 22:34:10.342126 15430 catalog_manager.cc:4841] TS eb245fc55cb0481584e61bdf8204759a (127.18.211.129:43255): tablet 21372da0158e45beae96db660f2505fd (table test-table [id=9a3801c2f89d4d149ee8d83c8bf8f7be]) successfully deleted
W20260811 22:34:10.422587 15773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:10.422869 15773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:10.422945 15773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:10.427368 15773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:10.427449 15773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:10.427537 15773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:10.432701 15773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:41071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=45735
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:10.433792 15773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:10.434777 15773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:10.444583 15787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:10.444572 15791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:10.444769 15773 server_base.cc:1034] running on GCE node
W20260811 22:34:10.444661 15788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:10.445343 15773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:10.446031 15773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:10.447248 15773 hybrid_clock.cc:648] HybridClock initialized: now 1786487650447216 us; error 45 us; skew 500 ppm
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:10.449913 15773 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:34:10.450789 15773 webserver.cc:460] Webserver started at http://127.18.211.130:45735/ using document root <none> and password file <none>
I20260811 22:34:10.451333 15773 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:10.451443 15773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:10.455495 15773 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:34:10.457614 15805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.458824 15773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:10.458976 15773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "a9eb3ae72a364d478d5c449e22cedcb8"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "2ad665f20c423b6aefb7c0a2bf67ae52"
server_key_iv: "df070acae57d967cd7e6faf4386e9043"
server_key_version: "encryptionkey@0"
I20260811 22:34:10.459511 15773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:10.479611 15773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:10.482724 15773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:10.482965 15773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:10.483598 15773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:10.484508 15773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:10.484586 15773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.484663 15773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:10.484709 15773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.494776 15773 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:41071
I20260811 22:34:10.494797 15994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:41071 every 8 connection(s)
I20260811 22:34:10.495903 15773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:10.506201 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 15773
I20260811 22:34:10.506709 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:46783
--local_ip_for_outbound_sockets=127.18.211.131
--tserver_master_addrs=127.18.211.190:45415
--webserver_port=34565
--webserver_interface=127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:34:10.509549 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.497886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:42781 (local address 127.18.211.190:45415)
0811 22:34:10.498031 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.498035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.499080 (+ 1045us) server_negotiation.cc:407] Connection header received
0811 22:34:10.499970 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.499973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.500038 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.500164 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:10.501398 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.501941 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.503081 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.503332 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.503436 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.505948 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:10.505952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:10.505954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:10.505977 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:10.507472 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.507976 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.507980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.507981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.508063 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.508422 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.508426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.508428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.508604 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:10.508712 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.509208 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.509378 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:34:10.510337 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.498262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:42781)
0811 22:34:10.498923 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.498963 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.499768 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.500358 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.500365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.500732 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:10.501203 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.501260 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.502069 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.502073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.502940 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.502949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.503487 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.503491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.503664 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.504263 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:10.504278 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:10.505733 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:10.507598 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.507604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.507617 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.507871 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.508171 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.508174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.508176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.508326 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.508712 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:10.508718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:10.508946 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.509161 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.509490 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":120,"threads_started":1}
I20260811 22:34:10.511487 15995 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45415
I20260811 22:34:10.511762 15995 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:10.512387 15995 heartbeater.cc:511] Master 127.18.211.190:45415 requested a full tablet report, sending...
I20260811 22:34:10.513571 15454 ts_manager.cc:194] Registered new tserver with Master: a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071)
I20260811 22:34:10.514226 15454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:42781
W20260811 22:34:10.619957 15998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:10.620190 15998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:10.620231 15998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:10.624203 15998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:10.624274 15998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:10.624372 15998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:10.629715 15998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:46783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=34565
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:10.630697 15998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:10.631682 15998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:10.641752 16015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:10.641788 16011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:10.641763 16012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:10.642241 15998 server_base.cc:1034] running on GCE node
I20260811 22:34:10.642637 15998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:10.643276 15998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:10.644459 15998 hybrid_clock.cc:648] HybridClock initialized: now 1786487650644444 us; error 26 us; skew 500 ppm
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:10.647310 15998 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:34:10.648128 15998 webserver.cc:460] Webserver started at http://127.18.211.131:34565/ using document root <none> and password file <none>
I20260811 22:34:10.648636 15998 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:10.648705 15998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:10.652580 15998 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260811 22:34:10.654809 16025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.655929 15998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:34:10.656062 15998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "4246ee5c3e94441580802a72bb3bec71"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "6b3846e4e290d10887c346dcb812bee8"
server_key_iv: "6ee7474715e0d8b7b572704cdda445f6"
server_key_version: "encryptionkey@0"
I20260811 22:34:10.656591 15998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:10.675530 15998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:10.678903 15998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:10.679133 15998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:10.679744 15998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:10.680583 15998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:10.680663 15998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.680735 15998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:10.680787 15998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:10.689682 15998 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:46783
I20260811 22:34:10.689747 16215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:46783 every 8 connection(s)
I20260811 22:34:10.690737 15998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:10.694705 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 15998
Aug 11 22:34:10 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:10.704391 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.695805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.18.211.190:45415)
0811 22:34:10.696078 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.696083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.696096 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:34:10.696135 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.696138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.696178 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.696274 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:10.697320 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.697946 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.698809 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.699035 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.699124 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.700751 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:10.700758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:10.700762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:10.700795 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:10.702497 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.703046 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.703051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.703053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.703142 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.703454 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.703456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.703457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.703621 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:10.703814 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.704051 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.704178 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":99,"threads_started":1}
I20260811 22:34:10.706073 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.692638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:45775 (local address 127.18.211.190:45415)
0811 22:34:10.692870 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.692873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.693910 (+ 1037us) server_negotiation.cc:407] Connection header received
0811 22:34:10.695071 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.695074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.695159 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.695273 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:10.696674 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.697555 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.698943 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.699238 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.699354 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.701994 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:10.701997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:10.701999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:10.702033 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:10.703706 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.704309 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.704312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.704314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.704396 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.704789 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.704792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.704794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.705002 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:10.705104 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.705826 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.705935 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 22:34:10.706974 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.693042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:45775)
0811 22:34:10.693752 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.693793 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.694824 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.695484 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.695491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.695857 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:10.696513 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.696528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.697686 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.697690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.698692 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.698702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.699408 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.699411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.699643 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.700223 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:10.700239 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:10.701812 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:10.703827 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.703835 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.703857 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.704216 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.704499 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:10.704503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:10.704505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:10.704687 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:10.705144 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:10.705151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:10.705517 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.705755 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.706013 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":179,"threads_started":1}
I20260811 22:34:10.708058 16217 heartbeater.cc:347] Connected to a master server at 127.18.211.190:45415
I20260811 22:34:10.708284 16217 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:10.708745 16217 heartbeater.cc:511] Master 127.18.211.190:45415 requested a full tablet report, sending...
I20260811 22:34:10.709741 15454 ts_manager.cc:194] Registered new tserver with Master: 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
I20260811 22:34:10.710286 15454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:45775
I20260811 22:34:10.718304 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:34:10.722170 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.718949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.18.211.190:45415)
0811 22:34:10.719069 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.719073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.719285 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:34:10.719418 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.719420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.719516 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.719600 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:10.719811 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.720294 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.721104 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.721381 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.721462 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.721732 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:10.721865 (+ 133us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 22:34:10.722064 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36810: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 22:34:10.722255 15772 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36810: token expired
W20260811 22:34:10.722615 15335 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.211.190:45415)
I20260811 22:34:10.729933 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.722978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.18.211.190:45415)
0811 22:34:10.723086 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.723089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.723100 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:34:10.723179 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.723181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.723215 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.723276 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:10.723836 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.724349 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.725082 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.725391 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.725465 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.726800 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:10.726802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:10.726804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:10.726821 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:10.728305 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.728862 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.728866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.728867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.728966 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:10.729272 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:10.729275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:10.729277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:10.729450 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:10.729554 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.729737 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.729826 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:34:10.732438 15454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:34:10.735756 15454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:34:10.743978 15651 tablet_service.cc:1467] Processing CreateTablet for tablet ad8012465bd045b48d50643cb65c2ee4 (DEFAULT_TABLE table=test-table [id=218b06a8de394d9fa0a67837b5b6bd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:10.744709 15651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8012465bd045b48d50643cb65c2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:10.747710 15579 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Bootstrap starting.
I20260811 22:34:10.748562 15579 tablet_bootstrap.cc:654] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:10.748977 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.743069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.18.211.130:41071)
0811 22:34:10.743445 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.743457 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.743480 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:34:10.743565 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.743571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.743827 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.744318 (+ 491us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:10.744507 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.745939 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.747551 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.748321 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.748439 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.748575 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.748784 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.748863 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":141,"threads_started":1}
I20260811 22:34:10.749033 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.742868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41071 (local address 127.0.0.1:46056)
0811 22:34:10.742996 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.743017 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.743084 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.744018 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.744021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.744041 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:10.744146 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.744156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.746051 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.746054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.747429 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.747437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.748457 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.748463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.748588 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.748654 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.748754 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.748855 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":175}
I20260811 22:34:10.749693 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.743329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:46783 (local address 127.0.0.1:50016)
0811 22:34:10.743626 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.743639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.743734 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.744539 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.744545 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.744567 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:10.744665 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.744672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.745942 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.745948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.747275 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.747287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.749204 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.749232 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.749328 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.749356 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.749447 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.749535 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":22,"thread_start_us":98,"threads_started":1}
I20260811 22:34:10.749816 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.743646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50016 (local address 127.18.211.131:46783)
0811 22:34:10.744039 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.744049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.744070 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:34:10.744139 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.744143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.744366 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.744842 (+ 476us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:10.745025 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.745830 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.747418 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.748307 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.748439 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.748575 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.749507 (+ 932us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.749639 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":176,"threads_started":1}
I20260811 22:34:10.750308 15579 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: No bootstrap required, opened a new log
I20260811 22:34:10.750419 15579 ts_tablet_manager.cc:1397] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:34:10.750922 15579 raft_consensus.cc:348] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.751055 15579 raft_consensus.cc:374] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:10.751101 15579 raft_consensus.cc:729] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb245fc55cb0481584e61bdf8204759a, State: Initialized, Role: FOLLOWER
I20260811 22:34:10.751281 15579 consensus_queue.cc:260] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.751591 15579 ts_tablet_manager.cc:1428] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:34:10.752935 15881 tablet_service.cc:1467] Processing CreateTablet for tablet ad8012465bd045b48d50643cb65c2ee4 (DEFAULT_TABLE table=test-table [id=218b06a8de394d9fa0a67837b5b6bd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:10.752935 16110 tablet_service.cc:1467] Processing CreateTablet for tablet ad8012465bd045b48d50643cb65c2ee4 (DEFAULT_TABLE table=test-table [id=218b06a8de394d9fa0a67837b5b6bd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:10.754112 16110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8012465bd045b48d50643cb65c2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:10.754112 15881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8012465bd045b48d50643cb65c2ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:10.762601 16233 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Bootstrap starting.
I20260811 22:34:10.765180 16233 tablet_bootstrap.cc:654] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:10.765591 16235 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Bootstrap starting.
I20260811 22:34:10.766323 16233 log.cc:824] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:10.768205 16235 tablet_bootstrap.cc:654] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:10.768631 16233 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: No bootstrap required, opened a new log
I20260811 22:34:10.768843 16233 ts_tablet_manager.cc:1397] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:34:10.769043 16235 log.cc:824] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:10.771047 16235 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: No bootstrap required, opened a new log
I20260811 22:34:10.771252 16235 ts_tablet_manager.cc:1397] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:34:10.772845 16233 raft_consensus.cc:348] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.773142 16233 raft_consensus.cc:374] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:10.773229 16233 raft_consensus.cc:729] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9eb3ae72a364d478d5c449e22cedcb8, State: Initialized, Role: FOLLOWER
I20260811 22:34:10.773715 16233 consensus_queue.cc:260] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.774389 15995 heartbeater.cc:503] Master 127.18.211.190:45415 was elected leader, sending a full tablet report...
I20260811 22:34:10.774777 16233 ts_tablet_manager.cc:1428] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:34:10.775020 16235 raft_consensus.cc:348] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.775254 16235 raft_consensus.cc:374] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:10.775337 16235 raft_consensus.cc:729] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4246ee5c3e94441580802a72bb3bec71, State: Initialized, Role: FOLLOWER
I20260811 22:34:10.775729 16235 consensus_queue.cc:260] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.776393 16217 heartbeater.cc:503] Master 127.18.211.190:45415 was elected leader, sending a full tablet report...
I20260811 22:34:10.776595 16235 ts_tablet_manager.cc:1428] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 22:34:10.794948 15766 tablet.cc:2369] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:34:10.914237 15777 raft_consensus.cc:482] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:10.914516 15777 raft_consensus.cc:504] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.915753 15777 leader_election.cc:288] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
I20260811 22:34:10.919890 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.916177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:57621 (local address 127.18.211.131:46783)
0811 22:34:10.916317 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.916321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.916563 (+ 242us) server_negotiation.cc:407] Connection header received
0811 22:34:10.916823 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.916826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.916890 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.916974 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:10.917247 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.917759 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.918901 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.919299 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.919430 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.919485 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.919682 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.919755 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:34:10.919893 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.916269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:47351 (local address 127.18.211.130:41071)
0811 22:34:10.916415 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.916420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.916624 (+ 204us) server_negotiation.cc:407] Connection header received
0811 22:34:10.916783 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.916786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.916832 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.916894 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:10.917259 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.917754 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.918903 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.919305 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.919441 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.919492 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.919723 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.919795 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:34:10.919912 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.916036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:46783 (local address 127.18.211.129:57621)
0811 22:34:10.916441 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.916456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.916609 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.917009 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.917011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.917031 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:10.917113 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.917118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.917901 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.917904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.918769 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.918779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.919430 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.919434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.919521 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.919572 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.919642 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.919741 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":17,"thread_start_us":100,"threads_started":1}
I20260811 22:34:10.919936 16244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.916036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41071 (local address 127.18.211.129:47351)
0811 22:34:10.916481 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:10.916494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:10.916639 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:10.916938 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:10.916942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:10.916964 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:10.917106 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.917119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.917877 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.917882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.918772 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:10.918781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.919458 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.919462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:10.919565 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.919594 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:34:10.919691 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 22:34:10.919757 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":116,"thread_start_us":84,"threads_started":1}
I20260811 22:34:10.920553 16142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8012465bd045b48d50643cb65c2ee4" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4246ee5c3e94441580802a72bb3bec71" is_pre_election: true
I20260811 22:34:10.920553 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8012465bd045b48d50643cb65c2ee4" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" is_pre_election: true
I20260811 22:34:10.920905 15920 raft_consensus.cc:2454] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 0.
I20260811 22:34:10.920905 16142 raft_consensus.cc:2454] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 0.
I20260811 22:34:10.921479 15576 leader_election.cc:302] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4246ee5c3e94441580802a72bb3bec71, eb245fc55cb0481584e61bdf8204759a; no voters:
I20260811 22:34:10.921794 15777 raft_consensus.cc:2781] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:34:10.921882 15777 raft_consensus.cc:482] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:34:10.921914 15777 raft_consensus.cc:3037] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:10.922876 15777 raft_consensus.cc:504] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.923237 15777 leader_election.cc:288] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 election: Requested vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
I20260811 22:34:10.923566 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8012465bd045b48d50643cb65c2ee4" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9eb3ae72a364d478d5c449e22cedcb8"
I20260811 22:34:10.923636 16142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8012465bd045b48d50643cb65c2ee4" candidate_uuid: "eb245fc55cb0481584e61bdf8204759a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4246ee5c3e94441580802a72bb3bec71"
I20260811 22:34:10.923741 16142 raft_consensus.cc:3037] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:10.923730 15920 raft_consensus.cc:3037] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:10.925107 16142 raft_consensus.cc:2454] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 1.
I20260811 22:34:10.925102 15920 raft_consensus.cc:2454] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb245fc55cb0481584e61bdf8204759a in term 1.
I20260811 22:34:10.925421 15575 leader_election.cc:302] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9eb3ae72a364d478d5c449e22cedcb8, eb245fc55cb0481584e61bdf8204759a; no voters:
I20260811 22:34:10.925586 15777 raft_consensus.cc:2781] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:10.925793 15777 raft_consensus.cc:686] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 LEADER]: Becoming Leader. State: Replica: eb245fc55cb0481584e61bdf8204759a, State: Running, Role: LEADER
I20260811 22:34:10.926071 15777 consensus_queue.cc:237] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:10.929199 15452 catalog_manager.cc:5495] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a reported cstate change: term changed from 0 to 1, leader changed from <none> to eb245fc55cb0481584e61bdf8204759a (127.18.211.129). New cstate: current_term: 1 leader_uuid: "eb245fc55cb0481584e61bdf8204759a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } health_report { overall_health: HEALTHY } } }
W20260811 22:34:10.941721 16220 tablet.cc:2369] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:34:10.983847 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:10.980389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.18.211.129:43255)
0811 22:34:10.980732 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:10.980739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:10.980752 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:34:10.980828 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:10.980830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:10.980883 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:10.980994 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:10.981313 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.981802 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.982579 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:10.982785 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:10.982870 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:10.983104 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:10.983517 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:34:10.983602 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:10.983676 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 22:34:10.983720 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":162,"threads_started":1}
I20260811 22:34:10.987861 16142 raft_consensus.cc:1261] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Refusing update from remote peer eb245fc55cb0481584e61bdf8204759a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:10.987864 15920 raft_consensus.cc:1261] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Refusing update from remote peer eb245fc55cb0481584e61bdf8204759a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:10.988631 16245 consensus_queue.cc:1035] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:10.988842 15777 consensus_queue.cc:1035] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:10.996197 16249 mvcc.cc:205] Tried to move back new op lower bound from 7317453418439634944 to 7317453418195795968. Current Snapshot: MvccSnapshot[applied={T|T < 7317453418439634944}]
I20260811 22:34:10.996969 16250 mvcc.cc:205] Tried to move back new op lower bound from 7317453418439634944 to 7317453418195795968. Current Snapshot: MvccSnapshot[applied={T|T < 7317453418439634944}]
I20260811 22:34:11.005271 15454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:11.005456 15454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:11.007968 15454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Sending DeleteTablet for 3 replicas of tablet ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:11.008692 15881 tablet_service.cc:1514] Processing DeleteTablet for tablet ad8012465bd045b48d50643cb65c2ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:11 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:46056
I20260811 22:34:11.008846 15651 tablet_service.cc:1514] Processing DeleteTablet for tablet ad8012465bd045b48d50643cb65c2ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:11 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:48756
I20260811 22:34:11.009133 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 15514
I20260811 22:34:11.009258 16110 tablet_service.cc:1514] Processing DeleteTablet for tablet ad8012465bd045b48d50643cb65c2ee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:11 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:50016
I20260811 22:34:11.019220 16268 tablet_replica.cc:330] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: stopping tablet replica
I20260811 22:34:11.019591 16268 raft_consensus.cc:2229] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:11.019942 16268 raft_consensus.cc:2258] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:11.022603 16268 ts_tablet_manager.cc:1905] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:11.022832 16265 tablet_replica.cc:330] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: stopping tablet replica
I20260811 22:34:11.023141 16265 raft_consensus.cc:2229] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:11.023444 16265 raft_consensus.cc:2258] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:11.025058 16265 ts_tablet_manager.cc:1905] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:11.025669 16268 ts_tablet_manager.cc:1918] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:11.025832 16268 log.cc:1196] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:11.026183 16268 ts_tablet_manager.cc:1939] T ad8012465bd045b48d50643cb65c2ee4 P 4246ee5c3e94441580802a72bb3bec71: Deleting consensus metadata
I20260811 22:34:11.027417 15433 catalog_manager.cc:4841] TS 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783): tablet ad8012465bd045b48d50643cb65c2ee4 (table test-table [id=218b06a8de394d9fa0a67837b5b6bd20]) successfully deleted
I20260811 22:34:11.028528 16265 ts_tablet_manager.cc:1918] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:11.028643 16265 log.cc:1196] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:11.028999 16265 ts_tablet_manager.cc:1939] T ad8012465bd045b48d50643cb65c2ee4 P a9eb3ae72a364d478d5c449e22cedcb8: Deleting consensus metadata
I20260811 22:34:11.030268 15432 catalog_manager.cc:4841] TS a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071): tablet ad8012465bd045b48d50643cb65c2ee4 (table test-table [id=218b06a8de394d9fa0a67837b5b6bd20]) successfully deleted
W20260811 22:34:11.031715 15430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:43255 (error 108)
I20260811 22:34:11.031975 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 15773
I20260811 22:34:11.035023 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:11.034724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:43255 (local address 127.0.0.1:48772)
0811 22:34:11.034860 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:11.034939 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:43255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:34:11.035207 15430 catalog_manager.cc:4584] TS eb245fc55cb0481584e61bdf8204759a (127.18.211.129:43255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad8012465bd045b48d50643cb65c2ee4: Network error: Client connection negotiation failed: client connection to 127.18.211.129:43255: connect: Connection refused (error 111)
I20260811 22:34:11.042724 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 15998
I20260811 22:34:11.058355 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 15398
I20260811 22:34:31.072310 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45415
--webserver_interface=127.18.211.190
--webserver_port=44369
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:45415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:31.195240 16270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:31.195519 16270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:31.195559 16270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:31.199570 16270 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:34:31.199635 16270 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:34:31.199651 16270 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:34:31.199664 16270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:31.199682 16270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:31.199700 16270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:31.199715 16270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:31.199730 16270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:31.205204 16270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:45415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:45415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=44369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:31.206321 16270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:31.207329 16270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:31.216635 16283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:31.216849 16270 server_base.cc:1034] running on GCE node
W20260811 22:34:31.216755 16279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:31.216725 16277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:31.217741 16270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:31.218871 16270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:31.220039 16270 hybrid_clock.cc:648] HybridClock initialized: now 1786487671220005 us; error 49 us; skew 500 ppm
Aug 11 22:34:31 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487671, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:31.228468 16270 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:34:31.229498 16270 webserver.cc:460] Webserver started at http://127.18.211.190:44369/ using document root <none> and password file <none>
I20260811 22:34:31.230080 16270 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:31.230145 16270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:31.234301 16270 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:34:31.236598 16298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.238080 16270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:34:31.238191 16270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "e2500f8bfe944e3e8880c653d577b433"
format_stamp: "Formatted at 2026-08-11 22:33:48 on dist-test-slave-8hhm"
server_key: "4d1ea61e1a72aeaa01202f641cc10230"
server_key_iv: "ef4f9b3249e42b91ec173395713df6ac"
server_key_version: "encryptionkey@0"
I20260811 22:34:31.238822 16270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:31.256184 16270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:31.259477 16270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:31.259701 16270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:31.267679 16270 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:45415
I20260811 22:34:31.268075 16389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:45415 every 8 connection(s)
I20260811 22:34:31.268886 16270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:31.271761 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 16270
I20260811 22:34:31.272221 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:43255
--local_ip_for_outbound_sockets=127.18.211.129
--tserver_master_addrs=127.18.211.190:45415
--webserver_port=42595
--webserver_interface=127.18.211.129
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:34:31.273599 16390 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:34:31.275326 16390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Bootstrap starting.
I20260811 22:34:31.301571 16390 log.cc:824] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:31.316815 16390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:34:31.317314 16390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Bootstrap complete.
I20260811 22:34:31.323145 16390 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:31.323699 16390 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2500f8bfe944e3e8880c653d577b433, State: Initialized, Role: FOLLOWER
I20260811 22:34:31.324323 16390 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:31.324481 16390 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:31.324565 16390 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:31.324666 16390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:34:31.327244 16390 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:31.327630 16390 leader_election.cc:302] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2500f8bfe944e3e8880c653d577b433; no voters:
I20260811 22:34:31.328003 16390 leader_election.cc:288] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:34:31.328152 16398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:34:31.328469 16398 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [term 3 LEADER]: Becoming Leader. State: Replica: e2500f8bfe944e3e8880c653d577b433, State: Running, Role: LEADER
I20260811 22:34:31.328866 16398 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } }
I20260811 22:34:31.329432 16390 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:31.330506 16399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e2500f8bfe944e3e8880c653d577b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } } }
I20260811 22:34:31.330506 16401 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2500f8bfe944e3e8880c653d577b433. Latest consensus state: current_term: 3 leader_uuid: "e2500f8bfe944e3e8880c653d577b433" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2500f8bfe944e3e8880c653d577b433" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 45415 } } }
I20260811 22:34:31.330660 16399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:31.330660 16401 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:31.331019 16411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:31.336395 16411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:31.337342 16411 catalog_manager.cc:1259] Loaded cluster ID: d1382cdf1afc494b9207716ab8e9597d
I20260811 22:34:31.337459 16411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:31.340132 16411 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:31.340910 16411 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Loaded TSK: 0
I20260811 22:34:31.341001 16411 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e2500f8bfe944e3e8880c653d577b433: Loaded TSK: 1
F20260811 22:34:31.341851 16411 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f810bf7c9ad google::LogMessage::Fail() at ??:0
@ 0x7f810bf7f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f810bf7c4de google::LogMessage::Flush() at ??:0
@ 0x7f810bf7da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f810ec4018d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f810ec38f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f81168e17bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f81168bb99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f81168bc178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f81168fddf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f811691c82b std::function<>::operator()() at ??:0
@ 0x7f81168bbb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f81168bcf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f81168b9579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f81168fd640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f810e1ce3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f810e1cec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f810e1d03f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f810e1bf512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f810f8036db start_thread at ??:0
@ 0x7f810aa7a61f clone at ??:0
W20260811 22:34:31.456425 16396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:31.456696 16396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:31.456770 16396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:31.461019 16396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:31.461086 16396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:31.461180 16396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:31.466679 16396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:43255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=42595
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:31.467777 16396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:31.468865 16396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:31.478878 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:31.478878 16467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:31.478907 16471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:31.480038 16396 server_base.cc:1034] running on GCE node
I20260811 22:34:31.480487 16396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:31.481266 16396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:31.482455 16396 hybrid_clock.cc:648] HybridClock initialized: now 1786487671482418 us; error 46 us; skew 500 ppm
Aug 11 22:34:31 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487671, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:31.485598 16396 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:34:31.486539 16396 webserver.cc:460] Webserver started at http://127.18.211.129:42595/ using document root <none> and password file <none>
I20260811 22:34:31.487144 16396 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:31.487244 16396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:31.491488 16396 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:34:31.493957 16480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.495404 16396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:34:31.495576 16396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "eb245fc55cb0481584e61bdf8204759a"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "5427122c4bd5faf73e4b4fe4c32e56a3"
server_key_iv: "1fea3377dfe0b3f9f9fbf4121f0fcb9b"
server_key_version: "encryptionkey@0"
I20260811 22:34:31.496184 16396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:31.517037 16396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:31.520711 16396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:31.520954 16396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:31.521751 16396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:31.522959 16489 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:34:31.528735 16396 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:34:31.528847 16396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 22:34:31.528944 16396 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:34:31.530860 16396 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:34:31.530944 16396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 22:34:31.531279 16489 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Bootstrap starting.
I20260811 22:34:31.542275 16396 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:43255
I20260811 22:34:31.542596 16672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:43255 every 8 connection(s)
I20260811 22:34:31.543468 16396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:31.552237 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 16396
I20260811 22:34:31.552721 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:41071
--local_ip_for_outbound_sockets=127.18.211.130
--tserver_master_addrs=127.18.211.190:45415
--webserver_port=45735
--webserver_interface=127.18.211.130
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:34:31.554101 16489 log.cc:824] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:31.564044 16489 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:34:31.564589 16489 tablet_bootstrap.cc:492] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Bootstrap complete.
I20260811 22:34:31.566094 16489 ts_tablet_manager.cc:1397] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Time spent bootstrapping tablet: real 0.035s user 0.026s sys 0.008s
I20260811 22:34:31.569518 16489 raft_consensus.cc:348] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:31.570119 16489 raft_consensus.cc:729] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb245fc55cb0481584e61bdf8204759a, State: Initialized, Role: FOLLOWER
I20260811 22:34:31.570688 16489 consensus_queue.cc:260] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:31.571835 16489 ts_tablet_manager.cc:1428] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260811 22:34:31.588409 16675 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:34:31.545750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:48703)
0811 22:34:31.546281 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.546339 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:31.547229 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:31.587587 (+ 40358us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":172,"threads_started":1}
W20260811 22:34:31.588918 16673 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:45415 (0 consecutive failures): Network error: Failed to ping master at 127.18.211.190:45415: Client connection negotiation failed: client connection to 127.18.211.190:45415: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:34:31.589767 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.589469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:49959)
0811 22:34:31.589621 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.589704 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 22:34:31.590458 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.590276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:58255)
0811 22:34:31.590371 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.590412 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:34:31.590693 16673 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:34:31.676455 16678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:31.676709 16678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:31.676753 16678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:31.680879 16678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:31.680944 16678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:31.681033 16678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:31.686271 16678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:41071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=45735
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:31.687376 16678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:31.688403 16678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:31.698436 16688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:31.698398 16692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:31.698537 16678 server_base.cc:1034] running on GCE node
W20260811 22:34:31.698467 16690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:31.699092 16678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:31.699812 16678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:31.701084 16678 hybrid_clock.cc:648] HybridClock initialized: now 1786487671701018 us; error 34 us; skew 500 ppm
Aug 11 22:34:31 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487671, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:31.703897 16678 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:34:31.704891 16678 webserver.cc:460] Webserver started at http://127.18.211.130:45735/ using document root <none> and password file <none>
I20260811 22:34:31.705545 16678 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:31.705633 16678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:31.709501 16678 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:34:31.711567 16701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.712805 16678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:31.712946 16678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "a9eb3ae72a364d478d5c449e22cedcb8"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "2ad665f20c423b6aefb7c0a2bf67ae52"
server_key_iv: "df070acae57d967cd7e6faf4386e9043"
server_key_version: "encryptionkey@0"
I20260811 22:34:31.713622 16678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:31.734045 16678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:31.737123 16678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:31.737373 16678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:31.738029 16678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:31.738914 16678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:31.738989 16678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.739063 16678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:31.739113 16678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.748931 16678 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:41071
I20260811 22:34:31.748957 16925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:41071 every 8 connection(s)
I20260811 22:34:31.750006 16678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:34:31.752305 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 16678
I20260811 22:34:31.752885 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:46783
--local_ip_for_outbound_sockets=127.18.211.131
--tserver_master_addrs=127.18.211.190:45415
--webserver_port=34565
--webserver_interface=127.18.211.131
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:34:31.754105 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.752204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:36921)
0811 22:34:31.752979 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.753107 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":179,"threads_started":1}
W20260811 22:34:31.754635 16927 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:45415 (0 consecutive failures): Network error: Failed to ping master at 127.18.211.190:45415: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
I20260811 22:34:31.755376 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.755067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:45939)
0811 22:34:31.755230 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.755299 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:34:31.756107 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.755840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:45407)
0811 22:34:31.755969 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.756028 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:34:31.756340 16927 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:34:31.868216 16932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:31.868454 16932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:31.868494 16932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:31.872586 16932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:31.872648 16932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:31.872756 16932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:31.877885 16932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:46783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=34565
--tserver_master_addrs=127.18.211.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:31.878923 16932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:31.879930 16932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:34:31.889849 16932 server_base.cc:1034] running on GCE node
W20260811 22:34:31.889760 16941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:31.889746 16942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:31.889761 16944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:31.890484 16932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:31.891091 16932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:31.892269 16932 hybrid_clock.cc:648] HybridClock initialized: now 1786487671892253 us; error 26 us; skew 500 ppm
Aug 11 22:34:31 dist-test-slave-8hhm krb5kdc[14445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487671, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:31.895378 16932 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:34:31.896263 16932 webserver.cc:460] Webserver started at http://127.18.211.131:34565/ using document root <none> and password file <none>
I20260811 22:34:31.896818 16932 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:31.896884 16932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:31.900746 16932 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:34:31.902940 16952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.904305 16932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:34:31.904487 16932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "4246ee5c3e94441580802a72bb3bec71"
format_stamp: "Formatted at 2026-08-11 22:33:49 on dist-test-slave-8hhm"
server_key: "6b3846e4e290d10887c346dcb812bee8"
server_key_iv: "6ee7474715e0d8b7b572704cdda445f6"
server_key_version: "encryptionkey@0"
I20260811 22:34:31.905196 16932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:31.914474 16932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:31.917433 16932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:31.917650 16932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:31.918280 16932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:31.919145 16932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:31.919224 16932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.919296 16932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:31.919353 16932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:31.929049 16932 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:46783
I20260811 22:34:31.929069 17152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:46783 every 8 connection(s)
I20260811 22:34:31.930127 16932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:31.931149 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 16932
I20260811 22:34:31.933990 17158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.932502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:57401)
0811 22:34:31.933151 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.933305 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":117,"threads_started":1}
W20260811 22:34:31.934449 17153 heartbeater.cc:644] Failed to heartbeat to 127.18.211.190:45415 (0 consecutive failures): Network error: Failed to ping master at 127.18.211.190:45415: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
I20260811 22:34:31.934996 17158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.934800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:59583)
0811 22:34:31.934898 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.934944 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 22:34:31.935679 17158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:31.935451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:59359)
0811 22:34:31.935585 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:31.935630 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 22:34:31.935927 17153 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:34:32.591902 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:32.591361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:47305)
0811 22:34:32.591736 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:32.591797 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260811 22:34:32.757400 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:32.756871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:34011)
0811 22:34:32.757175 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:32.757302 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":68,"threads_started":1}
I20260811 22:34:32.791761 17165 raft_consensus.cc:482] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:32.792002 17165 raft_consensus.cc:504] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:32.793182 17165 leader_election.cc:288] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
Aug 11 22:34:32 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487671, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.131@KRBTEST.COM
Aug 11 22:34:32 dist-test-slave-8hhm krb5kdc[14445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487671, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.130@KRBTEST.COM
I20260811 22:34:32.808043 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:32.793440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41071 (local address 127.18.211.129:59537)
0811 22:34:32.794282 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:32.794297 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:32.794411 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:32.796113 (+ 1702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:32.796116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:32.796342 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:32.796818 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:32.796834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.798190 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.798195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:32.799171 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:32.799181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.799921 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.799925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:32.800095 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:32.800851 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:32.800868 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:32.803063 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:32.805677 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:32.805681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:32.805683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:32.805922 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:32.806316 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:32.806319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:32.806321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:32.806459 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:32.807154 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:32.807164 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:32.807296 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 22:34:32.807487 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 22:34:32.807844 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":655,"spinlock_wait_cycles":7040,"thread_start_us":85,"threads_started":1}
I20260811 22:34:32.808113 17167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:32.793703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:59537 (local address 127.18.211.130:41071)
0811 22:34:32.794015 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:32.794043 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:32.794414 (+ 371us) server_negotiation.cc:407] Connection header received
0811 22:34:32.795096 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:32.795102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:32.795442 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:32.795731 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:32.797007 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.798039 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.799303 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.799800 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.799993 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:32.803317 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:32.803324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:32.803342 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:32.803380 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:32.805557 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:32.806071 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:32.806076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:32.806082 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:32.806210 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:32.806580 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:32.806583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:32.806585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:32.807030 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:32.807184 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:32.807605 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 22:34:32.807863 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":115,"threads_started":1}
I20260811 22:34:32.808152 17166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:32.793648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:59963 (local address 127.18.211.131:46783)
0811 22:34:32.794156 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:32.794171 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:32.794217 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:34:32.795086 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:32.795094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:32.795443 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:32.795770 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:32.797011 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.798063 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.799309 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.799839 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.800023 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:32.802917 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:32.802922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:32.802935 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:32.802963 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:32.805388 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:32.805954 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:32.805961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:32.805967 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:32.806095 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:32.806463 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:32.806468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:32.806470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:32.806921 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:32.807074 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:32.807604 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 22:34:32.807961 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":131,"threads_started":1}
I20260811 22:34:32.808161 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:32.793440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:46783 (local address 127.18.211.129:59963)
0811 22:34:32.793631 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:32.793646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:32.793754 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:32.795833 (+ 2079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:32.795839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:32.796340 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:32.796816 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:32.796832 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.798224 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.798226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:32.799172 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:32.799182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:32.799977 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:32.799981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:32.800095 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:32.800957 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:32.800978 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:32.802646 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:32.805518 (+ 2872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:32.805524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:32.805545 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:32.805830 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:32.806221 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:32.806224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:32.806226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:32.806360 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:32.807050 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:32.807057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:32.807289 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 22:34:32.807487 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 22:34:32.807961 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":398,"spinlock_wait_cycles":43136}
W20260811 22:34:32.811120 16483 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
W20260811 22:34:32.811786 16485 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:32.811858 16485 leader_election.cc:302] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb245fc55cb0481584e61bdf8204759a; no voters: 4246ee5c3e94441580802a72bb3bec71, a9eb3ae72a364d478d5c449e22cedcb8
I20260811 22:34:32.812153 17165 raft_consensus.cc:2726] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:34:32.936930 17169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:32.936534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:41709)
0811 22:34:32.936784 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:32.936836 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":69,"threads_started":1}
I20260811 22:34:33.593348 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:33.592796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:60195)
0811 22:34:33.593117 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:33.593191 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
I20260811 22:34:33.758621 17174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:33.758224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:38673)
0811 22:34:33.758484 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:33.758537 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260811 22:34:33.938091 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:33.937692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:40853)
0811 22:34:33.937932 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:33.937986 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260811 22:34:34.594708 17180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:34.594209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:38237)
0811 22:34:34.594538 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:34.594605 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":97,"threads_started":1}
I20260811 22:34:34.759873 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:34.759429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:52247)
0811 22:34:34.759719 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:34.759779 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
I20260811 22:34:34.908326 17182 raft_consensus.cc:482] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:34.908567 17182 raft_consensus.cc:504] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:34.909080 17182 leader_election.cc:288] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
W20260811 22:34:34.910029 16483 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
W20260811 22:34:34.910175 16485 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:34.910259 16485 leader_election.cc:302] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb245fc55cb0481584e61bdf8204759a; no voters: 4246ee5c3e94441580802a72bb3bec71, a9eb3ae72a364d478d5c449e22cedcb8
I20260811 22:34:34.910400 17182 raft_consensus.cc:2726] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:34:34.939319 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:34.938850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:51019)
0811 22:34:34.939166 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:34.939223 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":91,"threads_started":1}
I20260811 22:34:35.596176 17186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:35.595607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:33713)
0811 22:34:35.595950 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:35.596070 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":86,"threads_started":1}
I20260811 22:34:35.761106 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:35.760714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:41915)
0811 22:34:35.760961 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:35.761017 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260811 22:34:35.940577 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:35.940113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:34011)
0811 22:34:35.940423 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:35.940484 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":93,"threads_started":1}
I20260811 22:34:36.597572 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:36.597131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:42849)
0811 22:34:36.597423 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:36.597477 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":125,"threads_started":1}
I20260811 22:34:36.762358 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:36.761931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:34343)
0811 22:34:36.762188 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:36.762263 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
I20260811 22:34:36.941862 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:36.941419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:41583)
0811 22:34:36.941717 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:36.941770 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":132,"threads_started":1}
I20260811 22:34:37.598846 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:37.598404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:47365)
0811 22:34:37.598677 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:37.598737 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260811 22:34:37.763491 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:37.763099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:58399)
0811 22:34:37.763352 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:37.763403 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260811 22:34:37.829936 17198 raft_consensus.cc:482] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:37.830122 17198 raft_consensus.cc:504] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:37.830632 17198 leader_election.cc:288] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
W20260811 22:34:37.831449 16483 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
W20260811 22:34:37.831557 16485 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:37.831710 16485 leader_election.cc:302] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb245fc55cb0481584e61bdf8204759a; no voters: 4246ee5c3e94441580802a72bb3bec71, a9eb3ae72a364d478d5c449e22cedcb8
I20260811 22:34:37.831874 17198 raft_consensus.cc:2726] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:34:37.943073 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:37.942639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:47011)
0811 22:34:37.942919 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:37.942980 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":88,"threads_started":1}
I20260811 22:34:38.600224 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:38.599639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:47381)
0811 22:34:38.599909 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:38.599966 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
I20260811 22:34:38.764834 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:38.764363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:58945)
0811 22:34:38.764663 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:38.764740 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":75,"threads_started":1}
I20260811 22:34:38.944305 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:38.943845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:36185)
0811 22:34:38.944156 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:38.944210 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":91,"threads_started":1}
I20260811 22:34:39.449329 17212 raft_consensus.cc:482] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:39.449517 17212 raft_consensus.cc:504] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:39.450001 17212 leader_election.cc:288] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
W20260811 22:34:39.450825 16483 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
W20260811 22:34:39.450920 16485 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:39.451040 16485 leader_election.cc:302] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb245fc55cb0481584e61bdf8204759a; no voters: 4246ee5c3e94441580802a72bb3bec71, a9eb3ae72a364d478d5c449e22cedcb8
I20260811 22:34:39.451200 17212 raft_consensus.cc:2726] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:34:39.601519 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:39.600978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:46885)
0811 22:34:39.601305 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:39.601367 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
I20260811 22:34:39.766047 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:39.765602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:42121)
0811 22:34:39.765878 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:39.765954 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
I20260811 22:34:39.945554 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:39.945076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:34971)
0811 22:34:39.945390 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:39.945452 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":87,"threads_started":1}
I20260811 22:34:40.602735 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:40.602289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:55649)
0811 22:34:40.602572 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:40.602636 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":88,"threads_started":1}
I20260811 22:34:40.767331 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:40.766855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:41617)
0811 22:34:40.767153 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:40.767231 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":100,"threads_started":1}
I20260811 22:34:40.946744 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:40.946314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:58767)
0811 22:34:40.946574 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:40.946646 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260811 22:34:41.604130 17227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:41.603620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:49565)
0811 22:34:41.603906 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:41.603998 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":115,"threads_started":1}
I20260811 22:34:41.768548 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:41.768174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:42041)
0811 22:34:41.768408 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:41.768459 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260811 22:34:41.947985 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:41.947551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:44969)
0811 22:34:41.947828 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:41.947887 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
I20260811 22:34:42.534595 17237 raft_consensus.cc:482] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:42.534778 17237 raft_consensus.cc:504] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9eb3ae72a364d478d5c449e22cedcb8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41071 } } peers { permanent_uuid: "4246ee5c3e94441580802a72bb3bec71" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 46783 } } peers { permanent_uuid: "eb245fc55cb0481584e61bdf8204759a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43255 } }
I20260811 22:34:42.535248 17237 leader_election.cc:288] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071), 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783)
W20260811 22:34:42.536080 16485 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4246ee5c3e94441580802a72bb3bec71 (127.18.211.131:46783): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
W20260811 22:34:42.536187 16483 leader_election.cc:341] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a9eb3ae72a364d478d5c449e22cedcb8 (127.18.211.130:41071): Not found: Tablet not found: ad8012465bd045b48d50643cb65c2ee4
I20260811 22:34:42.536320 16483 leader_election.cc:302] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb245fc55cb0481584e61bdf8204759a; no voters: 4246ee5c3e94441580802a72bb3bec71, a9eb3ae72a364d478d5c449e22cedcb8
I20260811 22:34:42.536517 17237 raft_consensus.cc:2726] T ad8012465bd045b48d50643cb65c2ee4 P eb245fc55cb0481584e61bdf8204759a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:34:42.605504 17240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:42.605014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:59421)
0811 22:34:42.605342 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:42.605407 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":75,"threads_started":1}
I20260811 22:34:42.769780 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:42.769361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:56199)
0811 22:34:42.769625 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:42.769687 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
I20260811 22:34:42.949174 17242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:42.948738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:60833)
0811 22:34:42.949021 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:42.949082 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260811 22:34:43.606889 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:43.606423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:51553)
0811 22:34:43.606732 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:43.606792 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":102,"threads_started":1}
I20260811 22:34:43.770942 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:43.770516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:37895)
0811 22:34:43.770783 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:43.770840 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
I20260811 22:34:43.950382 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:43.949949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:34611)
0811 22:34:43.950224 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:43.950283 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260811 22:34:44.608299 17248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:44.607804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:33673)
0811 22:34:44.608126 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:44.608194 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":70,"threads_started":1}
I20260811 22:34:44.772085 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:44.771635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:35537)
0811 22:34:44.771927 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:44.771985 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":93,"threads_started":1}
I20260811 22:34:44.951668 17251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:44.951203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:59615)
0811 22:34:44.951496 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:44.951563 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":75,"threads_started":1}
I20260811 22:34:45.609820 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:45.609280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:41965)
0811 22:34:45.609655 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:45.609718 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
I20260811 22:34:45.773367 17255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:45.772886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:46867)
0811 22:34:45.773143 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:45.773201 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260811 22:34:45.952883 17257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:45.952447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:39365)
0811 22:34:45.952726 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:45.952784 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260811 22:34:46.611157 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:46.610760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.129:45823)
0811 22:34:46.611008 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:46.611062 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":84,"threads_started":1}
I20260811 22:34:46.774725 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:46.774326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.130:37919)
0811 22:34:46.774578 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:46.774629 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
I20260811 22:34:46.936640 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 16396
I20260811 22:34:46.954454 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:46.953985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:45415 (local address 127.18.211.131:55687)
0811 22:34:46.954279 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:46.954348 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.190:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
I20260811 22:34:46.957334 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 16678
I20260811 22:34:46.968134 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 16932
2026-08-11T22:34:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59743 ms)
[----------] 30 tests from SecurityITest (253345 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:34:47 dist-test-slave-8hhm krb5kdc[17268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:34:47 dist-test-slave-8hhm krb5kdc[17268](info): set up 2 sockets
Aug 11 22:34:47 dist-test-slave-8hhm krb5kdc[17268](info): commencing operation
krb5kdc: starting...
W20260811 22:34:48.282028 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:34:48 dist-test-slave-8hhm krb5kdc[17268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:34:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:34:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:34:48.416019 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39545
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:37083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:48.534190 17287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:48.534489 17287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:48.534531 17287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:48.538609 17287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:48.538682 17287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:48.538702 17287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:48.538719 17287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:48.538735 17287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:48.544258 17287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:39545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:48.545401 17287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:48.546486 17287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:48.555586 17297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:48.555657 17293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:48.555665 17294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:48.555729 17287 server_base.cc:1034] running on GCE node
I20260811 22:34:48.556613 17287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:48.557792 17287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:48.558950 17287 hybrid_clock.cc:648] HybridClock initialized: now 1786487688558922 us; error 39 us; skew 500 ppm
Aug 11 22:34:48 dist-test-slave-8hhm krb5kdc[17268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487688, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:48.567406 17287 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:34:48.568326 17287 webserver.cc:460] Webserver started at http://127.18.211.190:39263/ using document root <none> and password file <none>
I20260811 22:34:48.568922 17287 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:48.568987 17287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:48.569164 17287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:48.571319 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad"
format_stamp: "Formatted at 2026-08-11 22:34:48 on dist-test-slave-8hhm"
server_key: "460fe15e69952b26da8abeb913d13630"
server_key_iv: "a270ff4cbb9a70364c0798a46c14e7d7"
server_key_version: "encryptionkey@0"
I20260811 22:34:48.571933 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad"
format_stamp: "Formatted at 2026-08-11 22:34:48 on dist-test-slave-8hhm"
server_key: "460fe15e69952b26da8abeb913d13630"
server_key_iv: "a270ff4cbb9a70364c0798a46c14e7d7"
server_key_version: "encryptionkey@0"
I20260811 22:34:48.575935 17287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:34:48.578486 17310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:48.579737 17287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:34:48.579872 17287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad"
format_stamp: "Formatted at 2026-08-11 22:34:48 on dist-test-slave-8hhm"
server_key: "460fe15e69952b26da8abeb913d13630"
server_key_iv: "a270ff4cbb9a70364c0798a46c14e7d7"
server_key_version: "encryptionkey@0"
I20260811 22:34:48.579983 17287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:48.598629 17287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:48.602267 17287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:48.602519 17287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:48.610184 17287 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:39545
I20260811 22:34:48.610217 17399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:39545 every 8 connection(s)
I20260811 22:34:48.611229 17287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:48.613803 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 17287
I20260811 22:34:48.613951 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:34:48.614292 19278 external_mini_cluster.cc:1428] Setting key 6c25cb7443bf010cf0a0949339fb1c1a
I20260811 22:34:48.614415 17400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:48.621107 17400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad: Bootstrap starting.
Aug 11 22:34:48 dist-test-slave-8hhm krb5kdc[17268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:48.624092 17400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:48.625118 17400 log.cc:824] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:48.627605 17400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad: No bootstrap required, opened a new log
I20260811 22:34:48.629877 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:48.615839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.18.211.190:39545)
0811 22:34:48.616302 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:48.616317 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:48.616364 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:34:48.617204 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:48.617270 (+ 66us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:48.617613 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:48.617962 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:48.618807 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:48.619808 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:48.620685 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:48.621169 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:48.621428 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:48.624383 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:48.624391 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:48.624414 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:48.624451 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:48.626435 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:48.626892 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:48.626898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:48.626903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:48.627052 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:48.627336 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:48.627339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:48.627340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:48.627889 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:48.628090 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:48.628306 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 22:34:48.628628 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":155,"threads_started":1}
I20260811 22:34:48.631770 17400 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39545 } }
I20260811 22:34:48.632059 17400 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:48.632179 17400 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7e20fc9d0a49849ceee6ba015ef1ad, State: Initialized, Role: FOLLOWER
I20260811 22:34:48.632635 17400 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39545 } }
I20260811 22:34:48.632789 17400 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:48.632856 17400 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:48.632961 17400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:48.633978 17400 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39545 } }
I20260811 22:34:48.634344 17400 leader_election.cc:302] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff7e20fc9d0a49849ceee6ba015ef1ad; no voters:
I20260811 22:34:48.634678 17400 leader_election.cc:288] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:34:48.634847 17405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:48.635111 17405 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [term 1 LEADER]: Becoming Leader. State: Replica: ff7e20fc9d0a49849ceee6ba015ef1ad, State: Running, Role: LEADER
I20260811 22:34:48.635488 17405 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39545 } }
I20260811 22:34:48.636030 17400 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:48.637321 17407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39545 } } }
I20260811 22:34:48.637317 17411 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7e20fc9d0a49849ceee6ba015ef1ad. Latest consensus state: current_term: 1 leader_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e20fc9d0a49849ceee6ba015ef1ad" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 39545 } } }
I20260811 22:34:48.637480 17411 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:48.637480 17407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:48.637885 17428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:48.641250 17428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:48.647071 17428 catalog_manager.cc:1347] Generated new cluster ID: 3a01d0c880a8431792e0c5d004b02b87
I20260811 22:34:48.647150 17428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:48.658186 17428 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:34:48.659457 17428 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:48.669185 17428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad: Generated new TSK 0
I20260811 22:34:48.670081 17428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:34:48.708941 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39545
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:37083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:48.831907 17459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:48.832262 17459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:48.832329 17459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:48.836499 17459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:48.836561 17459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:48.836621 17459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:48.842226 17459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:48.843279 17459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:48.844262 17459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:48.854436 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:48.854436 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:48.854496 17459 server_base.cc:1034] running on GCE node
W20260811 22:34:48.854686 17475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:48.855074 17459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:48.855752 17459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:48.856938 17459 hybrid_clock.cc:648] HybridClock initialized: now 1786487688856905 us; error 46 us; skew 500 ppm
Aug 11 22:34:48 dist-test-slave-8hhm krb5kdc[17268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487688, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:48.859874 17459 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:34:48.860734 17459 webserver.cc:460] Webserver started at http://127.18.211.129:38273/ using document root <none> and password file <none>
I20260811 22:34:48.861342 17459 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:48.861413 17459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:48.861594 17459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:48.863623 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "552f9835f64c435bbeb60c7f7cd4c884"
format_stamp: "Formatted at 2026-08-11 22:34:48 on dist-test-slave-8hhm"
server_key: "e32aa3a431c573e69713986d3c3d2399"
server_key_iv: "0c26eeaa67aa4e449754f1495aa1fa27"
server_key_version: "encryptionkey@0"
I20260811 22:34:48.864156 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "552f9835f64c435bbeb60c7f7cd4c884"
format_stamp: "Formatted at 2026-08-11 22:34:48 on dist-test-slave-8hhm"
server_key: "e32aa3a431c573e69713986d3c3d2399"
server_key_iv: "0c26eeaa67aa4e449754f1495aa1fa27"
server_key_version: "encryptionkey@0"
I20260811 22:34:48.867869 17459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:34:48.870337 17484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:48.871642 17459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:48.871793 17459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "552f9835f64c435bbeb60c7f7cd4c884"
format_stamp: "Formatted at 2026-08-11 22:34:48 on dist-test-slave-8hhm"
server_key: "e32aa3a431c573e69713986d3c3d2399"
server_key_iv: "0c26eeaa67aa4e449754f1495aa1fa27"
server_key_version: "encryptionkey@0"
I20260811 22:34:48.871901 17459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:48.891680 17459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:48.894701 17459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:48.894938 17459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:48.895543 17459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:48.896435 17459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:48.896482 17459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:48.896567 17459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:48.896598 17459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:48.905750 17459 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:33595
I20260811 22:34:48.905774 17682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:33595 every 8 connection(s)
I20260811 22:34:48.906683 17459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:48.906993 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 17459
I20260811 22:34:48.907153 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:34:48.907545 19278 external_mini_cluster.cc:1428] Setting key c900898e1bef59ccbd39b247161709b3
Aug 11 22:34:48 dist-test-slave-8hhm krb5kdc[17268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487688, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:34:48.922384 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:48.908645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:33543 (local address 127.18.211.190:39545)
0811 22:34:48.908787 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:48.908791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:48.909787 (+ 996us) server_negotiation.cc:407] Connection header received
0811 22:34:48.911136 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:48.911139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:48.911193 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:48.911271 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:48.912933 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:48.913548 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:48.915415 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:48.915672 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:48.915756 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:48.918758 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:48.918761 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:48.918763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:48.918787 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:48.920251 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:48.920778 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:48.920782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:48.920784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:48.920865 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:48.921201 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:48.921204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:48.921206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:48.921459 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:48.921565 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:48.922065 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 22:34:48.922207 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:34:48.923183 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:48.909106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39545 (local address 127.18.211.129:33543)
0811 22:34:48.909643 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:48.909687 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:48.910918 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:48.911459 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:48.911470 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:48.912029 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:48.912763 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:48.912788 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:48.913679 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:48.913685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:48.915293 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:48.915308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:48.915851 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:48.915858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:48.916120 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:48.916805 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:48.916832 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:48.918580 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:48.920374 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:48.920380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:48.920395 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:48.920672 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:48.920956 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:48.920960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:48.920962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:48.921103 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:48.921566 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:48.921574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:48.921831 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 22:34:48.922020 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 22:34:48.922346 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":140,"threads_started":1}
I20260811 22:34:48.924481 17685 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39545
I20260811 22:34:48.924786 17685 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:48.925446 17685 heartbeater.cc:511] Master 127.18.211.190:39545 requested a full tablet report, sending...
I20260811 22:34:48.927330 17340 ts_manager.cc:194] Registered new tserver with Master: 552f9835f64c435bbeb60c7f7cd4c884 (127.18.211.129:33595)
I20260811 22:34:48.929119 17340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:33543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:34:48.940459 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39545
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:37083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:49.056077 17694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:49.056344 17694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:49.056417 17694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:49.060459 17694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:49.060518 17694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:49.060616 17694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:49.065903 17694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:49.066921 17694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:49.067938 17694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:34:49.077876 17694 server_base.cc:1034] running on GCE node
W20260811 22:34:49.077819 17706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:49.077849 17704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:49.077828 17709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:49.078485 17694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:49.079139 17694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:49.080331 17694 hybrid_clock.cc:648] HybridClock initialized: now 1786487689080294 us; error 49 us; skew 500 ppm
Aug 11 22:34:49 dist-test-slave-8hhm krb5kdc[17268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:49.083158 17694 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:34:49.084059 17694 webserver.cc:460] Webserver started at http://127.18.211.130:36639/ using document root <none> and password file <none>
I20260811 22:34:49.084640 17694 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:49.084733 17694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:49.084961 17694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:49.087091 17694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "ac48761e1abf419c94be54ad3e327ed4"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "f5d0cd8ac9630e1e49328acf9229d534"
server_key_iv: "0bfb4829b8e0ac8e03714aeaa138b335"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.087671 17694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "ac48761e1abf419c94be54ad3e327ed4"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "f5d0cd8ac9630e1e49328acf9229d534"
server_key_iv: "0bfb4829b8e0ac8e03714aeaa138b335"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.091410 17694 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:34:49.093930 17723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.095220 17694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:49.095372 17694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "ac48761e1abf419c94be54ad3e327ed4"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "f5d0cd8ac9630e1e49328acf9229d534"
server_key_iv: "0bfb4829b8e0ac8e03714aeaa138b335"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.095487 17694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:49.120365 17694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:49.123332 17694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:49.123548 17694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:49.124174 17694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:49.125021 17694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:49.125093 17694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.125164 17694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:49.125203 17694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.135941 17694 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:41023
I20260811 22:34:49.135962 17938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:41023 every 8 connection(s)
I20260811 22:34:49.137389 17694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:34:49.137596 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 17694
I20260811 22:34:49.137756 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:34:49.138116 19278 external_mini_cluster.cc:1428] Setting key dffae7a0e34924346318a0e5b803ff1e
Aug 11 22:34:49 dist-test-slave-8hhm krb5kdc[17268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:34:49.153705 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.139996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39545 (local address 127.18.211.130:35049)
0811 22:34:49.140466 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.140521 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:49.142110 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:49.142702 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:49.142713 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:49.143270 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:49.143908 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.143923 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.145100 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.145102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.146087 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.146098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.146808 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.146811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.146955 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.147577 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:49.147594 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:49.149071 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:49.150912 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:49.150917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:49.150928 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:49.151189 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:49.151436 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:49.151438 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:49.151439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:49.151573 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:49.151930 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:49.151936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:49.152238 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 22:34:49.152474 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 22:34:49.152791 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260811 22:34:49.154141 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.139535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:35049 (local address 127.18.211.190:39545)
0811 22:34:49.139673 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.139678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.140615 (+ 937us) server_negotiation.cc:407] Connection header received
0811 22:34:49.142369 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.142372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.142416 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.142500 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:49.144095 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.144945 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.146218 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.146653 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.146790 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.149268 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:49.149272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:49.149274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:49.149301 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:49.150803 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:49.151279 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:49.151282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:49.151283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:49.151359 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:49.151651 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:49.151653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:49.151654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:49.151836 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:49.151912 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.152532 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.152683 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:34:49.155310 17939 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39545
I20260811 22:34:49.155581 17939 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:49.156085 17939 heartbeater.cc:511] Master 127.18.211.190:39545 requested a full tablet report, sending...
I20260811 22:34:49.157205 17340 ts_manager.cc:194] Registered new tserver with Master: ac48761e1abf419c94be54ad3e327ed4 (127.18.211.130:41023)
I20260811 22:34:49.158154 17340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:35049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:34:49.171783 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39545
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:37083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:49.289007 17948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:49.289269 17948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:49.289310 17948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:49.293128 17948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:49.293184 17948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:49.293298 17948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:49.298530 17948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:39545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:49.299638 17948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:49.300695 17948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:49.310308 17958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:49.310434 17948 server_base.cc:1034] running on GCE node
W20260811 22:34:49.310341 17957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:49.310519 17960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:49.310976 17948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:49.311606 17948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:49.312767 17948 hybrid_clock.cc:648] HybridClock initialized: now 1786487689312752 us; error 27 us; skew 500 ppm
Aug 11 22:34:49 dist-test-slave-8hhm krb5kdc[17268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:49.315631 17948 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:34:49.316452 17948 webserver.cc:460] Webserver started at http://127.18.211.131:44183/ using document root <none> and password file <none>
I20260811 22:34:49.316967 17948 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:49.317025 17948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:49.317188 17948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:49.319259 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "09dbb4453262d1fb5b119534d79baaea"
server_key_iv: "126ec94d84ed784055e020a04cca48f6"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.319784 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "09dbb4453262d1fb5b119534d79baaea"
server_key_iv: "126ec94d84ed784055e020a04cca48f6"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.323446 17948 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 22:34:49.325848 17973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.327068 17948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:49.327212 17948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "09dbb4453262d1fb5b119534d79baaea"
server_key_iv: "126ec94d84ed784055e020a04cca48f6"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.327325 17948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:49.343062 17948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:49.346405 17948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:49.346619 17948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:49.347223 17948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:49.348057 17948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:49.348129 17948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.348204 17948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:49.348244 17948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.357484 17948 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:40205
I20260811 22:34:49.357510 18191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:40205 every 8 connection(s)
I20260811 22:34:49.358361 17948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:49.358660 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 17948
I20260811 22:34:49.358804 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:34:49.359172 19278 external_mini_cluster.cc:1428] Setting key 23f19e6f1848fbd1713bbf1efdb180c0
Aug 11 22:34:49 dist-test-slave-8hhm krb5kdc[17268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:49.372309 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.360248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:59383 (local address 127.18.211.190:39545)
0811 22:34:49.360394 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.360399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.361602 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 22:34:49.363001 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.363004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.363061 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.363138 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:49.364308 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.364845 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.365997 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.366216 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.366325 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.368678 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:49.368682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:49.368684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:49.368713 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:49.370174 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:49.370694 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:49.370697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:49.370700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:49.370782 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:49.371154 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:49.371159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:49.371161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:49.371347 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:49.371454 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.372019 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.372149 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:34:49.373297 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.360644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:39545 (local address 127.18.211.131:59383)
0811 22:34:49.361161 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.361202 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:49.362789 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:49.363317 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:49.363324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:49.363674 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:49.364163 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.364176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.364970 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.364973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.365863 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.365872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.366361 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.366365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.366543 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.367068 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:49.367084 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:49.368521 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:49.370304 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:49.370310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:49.370324 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:49.370587 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:49.370875 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:49.370879 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:49.370881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:49.371055 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:49.371454 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:49.371461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:49.371733 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 22:34:49.371943 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 22:34:49.372276 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":152,"threads_started":1}
I20260811 22:34:49.374380 18194 heartbeater.cc:347] Connected to a master server at 127.18.211.190:39545
I20260811 22:34:49.374630 18194 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:49.375104 18194 heartbeater.cc:511] Master 127.18.211.190:39545 requested a full tablet report, sending...
I20260811 22:34:49.376114 17340 ts_manager.cc:194] Registered new tserver with Master: 1ef0386d9bb24d0f8a971dd639fd4a5d (127.18.211.131:40205)
I20260811 22:34:49.376672 17340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:59383
I20260811 22:34:49.385026 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:34:49.393997 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.387097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.18.211.190:39545)
0811 22:34:49.387246 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.387251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.387389 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:34:49.387489 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.387491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.387530 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.387594 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:49.388231 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.388726 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.389572 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.389787 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.389865 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.391113 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:49.391116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:49.391118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:49.391141 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:49.392441 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:49.392819 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:49.392821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:49.392823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:49.392897 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:49.393127 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:49.393129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:49.393130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:49.393345 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:49.393450 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.393709 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.393802 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:34:49.397117 17340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:34:49.400370 17340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:34:49.414453 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.408764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:40205 (local address 127.0.0.1:47114)
0811 22:34:49.409100 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.409171 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:49.409360 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:49.410535 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:49.410541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:49.410565 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:49.410696 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.410703 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.411875 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.411881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.413072 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.413082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.413908 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.413911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.414033 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.414079 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:34:49.414199 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:34:49.414311 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
I20260811 22:34:49.414481 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.409621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.18.211.131:40205)
0811 22:34:49.410024 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.410038 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.410067 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:34:49.410147 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.410153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.410369 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.410544 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:49.410851 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.411744 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.413227 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.413787 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.413930 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.414072 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.414248 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.414341 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":97,"threads_started":1}
I20260811 22:34:49.415447 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.408797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:33595 (local address 127.0.0.1:45478)
0811 22:34:49.410050 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.410066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:49.410157 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:49.411005 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:49.411008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:49.411025 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:49.411106 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.411111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.412489 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.412492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.413743 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.413752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.415061 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.415065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.415141 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.415161 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:34:49.415239 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:34:49.415307 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"mutex_wait_us":61,"thread_start_us":101,"threads_started":1}
I20260811 22:34:49.415518 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.409819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.18.211.129:33595)
0811 22:34:49.410374 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.410387 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.410404 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:34:49.410489 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.410495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.410704 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.410863 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:49.411233 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.412355 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.413894 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.414769 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.414889 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.415012 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.415287 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.415378 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":71,"threads_started":1}
I20260811 22:34:49.416072 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.409737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.18.211.130:41023)
0811 22:34:49.410179 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.410194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.410386 (+ 192us) server_negotiation.cc:407] Connection header received
0811 22:34:49.410473 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.410480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.410672 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.410835 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:49.411072 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.412254 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.413878 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.414706 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.414847 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.414973 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.415847 (+ 874us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.415953 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":126,"threads_started":1}
I20260811 22:34:49.416316 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.409519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41023 (local address 127.0.0.1:42012)
0811 22:34:49.410161 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.410175 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:49.410284 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:49.410813 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:49.410816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:49.410835 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:49.410940 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.410947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.412420 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.412425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.413741 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.413752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.414836 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.414839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.414922 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.414944 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:34:49.415039 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:34:49.415108 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":27,"thread_start_us":82,"threads_started":1}
I20260811 22:34:49.418133 18070 tablet_service.cc:1467] Processing CreateTablet for tablet 325226f5a4894ad4926b760a45fc69e8 (DEFAULT_TABLE table=test-table [id=38af8b2b9b81404cb8d29c19ce8bc4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:49.418380 17577 tablet_service.cc:1467] Processing CreateTablet for tablet 325226f5a4894ad4926b760a45fc69e8 (DEFAULT_TABLE table=test-table [id=38af8b2b9b81404cb8d29c19ce8bc4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:49.418630 17808 tablet_service.cc:1467] Processing CreateTablet for tablet 325226f5a4894ad4926b760a45fc69e8 (DEFAULT_TABLE table=test-table [id=38af8b2b9b81404cb8d29c19ce8bc4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:49.419322 18070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325226f5a4894ad4926b760a45fc69e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:49.419322 17577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325226f5a4894ad4926b760a45fc69e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:49.419819 17808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325226f5a4894ad4926b760a45fc69e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:49.425806 18220 tablet_bootstrap.cc:492] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Bootstrap starting.
I20260811 22:34:49.427297 18225 tablet_bootstrap.cc:492] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Bootstrap starting.
I20260811 22:34:49.427606 18222 tablet_bootstrap.cc:492] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884: Bootstrap starting.
I20260811 22:34:49.428445 18220 tablet_bootstrap.cc:654] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:49.429438 18222 tablet_bootstrap.cc:654] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:49.429495 18225 tablet_bootstrap.cc:654] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:49.430334 18222 log.cc:824] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:49.430388 18220 log.cc:824] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:49.430792 18225 log.cc:824] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:49.432421 18220 tablet_bootstrap.cc:492] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: No bootstrap required, opened a new log
I20260811 22:34:49.432437 18222 tablet_bootstrap.cc:492] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884: No bootstrap required, opened a new log
I20260811 22:34:49.432629 18220 ts_tablet_manager.cc:1397] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 22:34:49.432634 18222 ts_tablet_manager.cc:1397] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:34:49.433547 18225 tablet_bootstrap.cc:492] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: No bootstrap required, opened a new log
I20260811 22:34:49.433732 18225 ts_tablet_manager.cc:1397] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:34:49.435678 18220 raft_consensus.cc:348] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.435998 18220 raft_consensus.cc:374] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:49.436084 18220 raft_consensus.cc:729] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac48761e1abf419c94be54ad3e327ed4, State: Initialized, Role: FOLLOWER
I20260811 22:34:49.436589 18220 consensus_queue.cc:260] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.437667 17939 heartbeater.cc:503] Master 127.18.211.190:39545 was elected leader, sending a full tablet report...
I20260811 22:34:49.437434 18222 raft_consensus.cc:348] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.437753 18220 ts_tablet_manager.cc:1428] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 22:34:49.437809 18222 raft_consensus.cc:374] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:49.437878 18222 raft_consensus.cc:729] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552f9835f64c435bbeb60c7f7cd4c884, State: Initialized, Role: FOLLOWER
I20260811 22:34:49.437866 18225 raft_consensus.cc:348] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.438120 18225 raft_consensus.cc:374] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:49.438218 18225 raft_consensus.cc:729] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef0386d9bb24d0f8a971dd639fd4a5d, State: Initialized, Role: FOLLOWER
I20260811 22:34:49.438476 18222 consensus_queue.cc:260] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.438676 18225 consensus_queue.cc:260] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.439558 18225 ts_tablet_manager.cc:1428] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:34:49.439550 18222 ts_tablet_manager.cc:1428] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:34:49.439955 18194 heartbeater.cc:503] Master 127.18.211.190:39545 was elected leader, sending a full tablet report...
I20260811 22:34:49.440668 17685 heartbeater.cc:503] Master 127.18.211.190:39545 was elected leader, sending a full tablet report...
I20260811 22:34:49.457638 18234 raft_consensus.cc:482] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:49.457811 18234 raft_consensus.cc:504] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.458833 18234 leader_election.cc:288] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ef0386d9bb24d0f8a971dd639fd4a5d (127.18.211.131:40205), ac48761e1abf419c94be54ad3e327ed4 (127.18.211.130:41023)
I20260811 22:34:49.463140 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.459205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:39809 (local address 127.18.211.131:40205)
0811 22:34:49.459342 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.459346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.459604 (+ 258us) server_negotiation.cc:407] Connection header received
0811 22:34:49.459901 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.459906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.459974 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.460052 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:49.460374 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.460946 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.462171 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.462577 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.462704 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.462761 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.462927 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.463005 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:34:49.463176 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.459169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:40205 (local address 127.18.211.129:39809)
0811 22:34:49.459465 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.459482 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:49.459616 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:49.460064 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:49.460066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:49.460104 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:49.460257 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.460268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.461047 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.461050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.462048 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.462058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.462697 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.462702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.462790 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.462823 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:34:49.462891 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:34:49.462967 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":5,"thread_start_us":131,"threads_started":1}
I20260811 22:34:49.463207 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.459774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:59511 (local address 127.18.211.130:41023)
0811 22:34:49.459920 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.459924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.459972 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:34:49.460150 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.460154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.460201 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.460289 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:49.460567 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.461081 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.462219 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.462606 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.462740 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.462793 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.463004 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.463095 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:34:49.463220 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.459591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:41023 (local address 127.18.211.129:59511)
0811 22:34:49.459894 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.459907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:49.460033 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:49.460307 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:49.460309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:49.460342 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:49.460454 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.460461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.461204 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.461245 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.462070 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:49.462079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.462753 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.462756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:49.462852 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.462888 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:34:49.462970 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:34:49.463032 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":27,"thread_start_us":95,"threads_started":1}
I20260811 22:34:49.463809 17847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325226f5a4894ad4926b760a45fc69e8" candidate_uuid: "552f9835f64c435bbeb60c7f7cd4c884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac48761e1abf419c94be54ad3e327ed4" is_pre_election: true
I20260811 22:34:49.463827 18110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325226f5a4894ad4926b760a45fc69e8" candidate_uuid: "552f9835f64c435bbeb60c7f7cd4c884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" is_pre_election: true
I20260811 22:34:49.464114 17847 raft_consensus.cc:2454] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552f9835f64c435bbeb60c7f7cd4c884 in term 0.
I20260811 22:34:49.464123 18110 raft_consensus.cc:2454] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552f9835f64c435bbeb60c7f7cd4c884 in term 0.
I20260811 22:34:49.464584 17487 leader_election.cc:302] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 552f9835f64c435bbeb60c7f7cd4c884, ac48761e1abf419c94be54ad3e327ed4; no voters:
I20260811 22:34:49.464872 18234 raft_consensus.cc:2781] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:34:49.464967 18234 raft_consensus.cc:482] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:34:49.465018 18234 raft_consensus.cc:3037] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:49.466101 18234 raft_consensus.cc:504] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.466436 18234 leader_election.cc:288] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [CANDIDATE]: Term 1 election: Requested vote from peers 1ef0386d9bb24d0f8a971dd639fd4a5d (127.18.211.131:40205), ac48761e1abf419c94be54ad3e327ed4 (127.18.211.130:41023)
I20260811 22:34:49.466758 18110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325226f5a4894ad4926b760a45fc69e8" candidate_uuid: "552f9835f64c435bbeb60c7f7cd4c884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d"
I20260811 22:34:49.466831 17847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325226f5a4894ad4926b760a45fc69e8" candidate_uuid: "552f9835f64c435bbeb60c7f7cd4c884" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac48761e1abf419c94be54ad3e327ed4"
I20260811 22:34:49.466869 18110 raft_consensus.cc:3037] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:49.466985 17847 raft_consensus.cc:3037] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:49.468034 18110 raft_consensus.cc:2454] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552f9835f64c435bbeb60c7f7cd4c884 in term 1.
I20260811 22:34:49.468222 17847 raft_consensus.cc:2454] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552f9835f64c435bbeb60c7f7cd4c884 in term 1.
I20260811 22:34:49.468324 17487 leader_election.cc:302] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef0386d9bb24d0f8a971dd639fd4a5d, 552f9835f64c435bbeb60c7f7cd4c884; no voters:
I20260811 22:34:49.468523 18234 raft_consensus.cc:2781] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:49.468744 18234 raft_consensus.cc:686] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [term 1 LEADER]: Becoming Leader. State: Replica: 552f9835f64c435bbeb60c7f7cd4c884, State: Running, Role: LEADER
I20260811 22:34:49.469084 18234 consensus_queue.cc:237] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } }
I20260811 22:34:49.472153 17336 catalog_manager.cc:5495] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 reported cstate change: term changed from 0 to 1, leader changed from <none> to 552f9835f64c435bbeb60c7f7cd4c884 (127.18.211.129). New cstate: current_term: 1 leader_uuid: "552f9835f64c435bbeb60c7f7cd4c884" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "552f9835f64c435bbeb60c7f7cd4c884" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 33595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 } health_report { overall_health: UNKNOWN } } }
I20260811 22:34:49.492647 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.489191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.18.211.129:33595)
0811 22:34:49.489402 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.489405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.489420 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:34:49.489518 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.489521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.489577 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.489671 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:49.489899 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.490414 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.491251 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.491464 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.491547 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.491866 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:49.492322 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:34:49.492423 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.492469 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.492522 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 22:34:49.498641 18110 raft_consensus.cc:1261] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 1 FOLLOWER]: Refusing update from remote peer 552f9835f64c435bbeb60c7f7cd4c884: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:49.498646 17847 raft_consensus.cc:1261] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 1 FOLLOWER]: Refusing update from remote peer 552f9835f64c435bbeb60c7f7cd4c884: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:49.499374 18234 consensus_queue.cc:1035] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef0386d9bb24d0f8a971dd639fd4a5d" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:49.499567 18240 consensus_queue.cc:1035] T 325226f5a4894ad4926b760a45fc69e8 P 552f9835f64c435bbeb60c7f7cd4c884 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac48761e1abf419c94be54ad3e327ed4" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:49.505797 18247 mvcc.cc:205] Tried to move back new op lower bound from 7317453576178556928 to 7317453576067977216. Current Snapshot: MvccSnapshot[applied={T|T < 7317453576178556928}]
I20260811 22:34:49.506928 18251 mvcc.cc:205] Tried to move back new op lower bound from 7317453576178556928 to 7317453576067977216. Current Snapshot: MvccSnapshot[applied={T|T < 7317453576178556928}]
I20260811 22:34:49.510162 18250 mvcc.cc:205] Tried to move back new op lower bound from 7317453576178556928 to 7317453576067977216. Current Snapshot: MvccSnapshot[applied={T|T < 7317453576178556928}]
I20260811 22:34:49.514763 17337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:49.514910 17337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:49.517340 17337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff7e20fc9d0a49849ceee6ba015ef1ad: Sending DeleteTablet for 3 replicas of tablet 325226f5a4894ad4926b760a45fc69e8
I20260811 22:34:49.518131 17808 tablet_service.cc:1514] Processing DeleteTablet for tablet 325226f5a4894ad4926b760a45fc69e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:42012
I20260811 22:34:49.518148 18070 tablet_service.cc:1514] Processing DeleteTablet for tablet 325226f5a4894ad4926b760a45fc69e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:47114
I20260811 22:34:49.518147 17577 tablet_service.cc:1514] Processing DeleteTablet for tablet 325226f5a4894ad4926b760a45fc69e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:49 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:45478
I20260811 22:34:49.518657 18267 tablet_replica.cc:330] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: stopping tablet replica
I20260811 22:34:49.518805 18274 tablet_replica.cc:330] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: stopping tablet replica
I20260811 22:34:49.519029 18267 raft_consensus.cc:2229] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:49.519102 18274 raft_consensus.cc:2229] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:49.519404 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 17459
I20260811 22:34:49.519421 18274 raft_consensus.cc:2258] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:49.519421 18267 raft_consensus.cc:2258] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:49.521070 18267 ts_tablet_manager.cc:1905] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:49.521070 18274 ts_tablet_manager.cc:1905] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:49.523838 18267 ts_tablet_manager.cc:1918] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:49.523837 18274 ts_tablet_manager.cc:1918] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:49.523942 18274 log.cc:1196] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/325226f5a4894ad4926b760a45fc69e8
I20260811 22:34:49.523979 18267 log.cc:1196] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/325226f5a4894ad4926b760a45fc69e8
I20260811 22:34:49.524330 18267 ts_tablet_manager.cc:1939] T 325226f5a4894ad4926b760a45fc69e8 P ac48761e1abf419c94be54ad3e327ed4: Deleting consensus metadata
I20260811 22:34:49.526007 18274 ts_tablet_manager.cc:1939] T 325226f5a4894ad4926b760a45fc69e8 P 1ef0386d9bb24d0f8a971dd639fd4a5d: Deleting consensus metadata
I20260811 22:34:49.528136 17311 catalog_manager.cc:4841] TS 1ef0386d9bb24d0f8a971dd639fd4a5d (127.18.211.131:40205): tablet 325226f5a4894ad4926b760a45fc69e8 (table test-table [id=38af8b2b9b81404cb8d29c19ce8bc4e2]) successfully deleted
I20260811 22:34:49.528578 17311 catalog_manager.cc:4841] TS ac48761e1abf419c94be54ad3e327ed4 (127.18.211.130:41023): tablet 325226f5a4894ad4926b760a45fc69e8 (table test-table [id=38af8b2b9b81404cb8d29c19ce8bc4e2]) successfully deleted
W20260811 22:34:49.534758 17314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:33595 (error 108)
I20260811 22:34:49.534941 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 17694
I20260811 22:34:49.536142 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.535702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:33595 (local address 127.0.0.1:45488)
0811 22:34:49.535935 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:49.536025 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:33595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260811 22:34:49.536375 17314 catalog_manager.cc:4584] TS 552f9835f64c435bbeb60c7f7cd4c884 (127.18.211.129:33595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 325226f5a4894ad4926b760a45fc69e8: Network error: Client connection negotiation failed: client connection to 127.18.211.129:33595: connect: Connection refused (error 111)
I20260811 22:34:49.549885 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 17948
I20260811 22:34:49.562525 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 17287
2026-08-11T22:34:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2595 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T22:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:34:49Z Disabled control of system clock
I20260811 22:34:49.601586 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:33707
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:34:49.716557 18281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:49.716948 18281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:49.716986 18281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:49.720898 18281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:49.720959 18281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:49.720974 18281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:49.720991 18281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:49.721006 18281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:49.725718 18281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:33707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:49.726768 18281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:49.728205 18281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:49.736756 18287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:49.736840 18293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:49.736858 18281 server_base.cc:1034] running on GCE node
W20260811 22:34:49.736756 18288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:49.737699 18281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:49.738742 18281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:49.739914 18281 hybrid_clock.cc:648] HybridClock initialized: now 1786487689739887 us; error 38 us; skew 500 ppm
I20260811 22:34:49.741813 18281 webserver.cc:460] Webserver started at http://127.18.211.190:41957/ using document root <none> and password file <none>
I20260811 22:34:49.742434 18281 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:49.742497 18281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:49.742755 18281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:49.744827 18281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "71243adbef1f4b9c9c423c274064c48f"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "773a7a3bba066f1991a418f8b77f4afc"
server_key_iv: "aca1f689db1d9d30076afc3b879f610e"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.745424 18281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "71243adbef1f4b9c9c423c274064c48f"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "773a7a3bba066f1991a418f8b77f4afc"
server_key_iv: "aca1f689db1d9d30076afc3b879f610e"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.748973 18281 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:34:49.751459 18302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.752660 18281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:49.752797 18281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "71243adbef1f4b9c9c423c274064c48f"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "773a7a3bba066f1991a418f8b77f4afc"
server_key_iv: "aca1f689db1d9d30076afc3b879f610e"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.752902 18281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:49.769872 18281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:49.770502 18281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:49.770704 18281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:49.777253 18281 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:33707
I20260811 22:34:49.777268 18395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:33707 every 8 connection(s)
I20260811 22:34:49.778173 18281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:49.781252 18396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:49.786779 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f: Bootstrap starting.
I20260811 22:34:49.787951 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 18281
I20260811 22:34:49.788089 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:34:49.788378 19278 external_mini_cluster.cc:1428] Setting key 5d105011902c4533bb8e32d29d5560d6
I20260811 22:34:49.789498 18396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:49.790366 18396 log.cc:824] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:49.792491 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f: No bootstrap required, opened a new log
I20260811 22:34:49.796885 18396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71243adbef1f4b9c9c423c274064c48f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33707 } }
I20260811 22:34:49.797108 18396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:49.797181 18396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71243adbef1f4b9c9c423c274064c48f, State: Initialized, Role: FOLLOWER
I20260811 22:34:49.797578 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:49.789575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.18.211.190:33707)
0811 22:34:49.790266 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:49.790279 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:49.790315 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:34:49.790965 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:49.790982 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:49.791313 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:49.791657 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:49.792034 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.793024 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.794280 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:49.794741 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:49.794868 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:49.795765 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:49.795769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:49.795783 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:49.796151 (+ 368us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:49.796206 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:49.796225 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:49.796301 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:49.796467 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:34:49.796758 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":160,"threads_started":1}
I20260811 22:34:49.797659 18396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71243adbef1f4b9c9c423c274064c48f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33707 } }
I20260811 22:34:49.797837 18396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:49.797884 18396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:49.797951 18396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:49.798885 18396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71243adbef1f4b9c9c423c274064c48f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33707 } }
I20260811 22:34:49.799257 18396 leader_election.cc:302] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71243adbef1f4b9c9c423c274064c48f; no voters:
I20260811 22:34:49.799582 18396 leader_election.cc:288] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:34:49.799685 18405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:49.799918 18405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [term 1 LEADER]: Becoming Leader. State: Replica: 71243adbef1f4b9c9c423c274064c48f, State: Running, Role: LEADER
I20260811 22:34:49.800208 18405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71243adbef1f4b9c9c423c274064c48f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33707 } }
I20260811 22:34:49.800812 18396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:49.801270 18405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71243adbef1f4b9c9c423c274064c48f. Latest consensus state: current_term: 1 leader_uuid: "71243adbef1f4b9c9c423c274064c48f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71243adbef1f4b9c9c423c274064c48f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33707 } } }
I20260811 22:34:49.801381 18405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:49.801810 18410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:49.801923 18406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71243adbef1f4b9c9c423c274064c48f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71243adbef1f4b9c9c423c274064c48f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 33707 } } }
I20260811 22:34:49.802114 18406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:49.804952 18410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:49.810655 18410 catalog_manager.cc:1347] Generated new cluster ID: 0227be04213e4ae3a8c20f56575130fb
I20260811 22:34:49.810730 18410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:49.824213 18410 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:34:49.825091 18410 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:49.834263 18410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f: Generated new TSK 0
I20260811 22:34:49.834959 18410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:34:49.838753 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33707
--builtin_ntp_servers=127.18.211.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:34:49.954715 18436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:49.954974 18436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:49.955011 18436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:49.959308 18436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:49.959370 18436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:49.959425 18436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:49.964165 18436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:49.965389 18436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:49.966840 18436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:49.976915 18456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:49.976954 18453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:49.976939 18445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:49.977485 18436 server_base.cc:1034] running on GCE node
I20260811 22:34:49.977962 18436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:49.978664 18436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:49.979854 18436 hybrid_clock.cc:648] HybridClock initialized: now 1786487689979832 us; error 37 us; skew 500 ppm
I20260811 22:34:49.981721 18436 webserver.cc:460] Webserver started at http://127.18.211.129:41543/ using document root <none> and password file <none>
I20260811 22:34:49.982334 18436 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:49.982427 18436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:49.982664 18436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:49.984783 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "a26f75a6027046a381e5475366bbd32a"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "712c0235d5c17f132a227073334013d9"
server_key_iv: "6b083e461b4bc1eba6ba2344b596f05e"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.985425 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "a26f75a6027046a381e5475366bbd32a"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "712c0235d5c17f132a227073334013d9"
server_key_iv: "6b083e461b4bc1eba6ba2344b596f05e"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.989022 18436 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:34:49.991590 18468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:49.992785 18436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:49.992925 18436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "a26f75a6027046a381e5475366bbd32a"
format_stamp: "Formatted at 2026-08-11 22:34:49 on dist-test-slave-8hhm"
server_key: "712c0235d5c17f132a227073334013d9"
server_key_iv: "6b083e461b4bc1eba6ba2344b596f05e"
server_key_version: "encryptionkey@0"
I20260811 22:34:49.993040 18436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:50.017627 18436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:50.018388 18436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:50.018600 18436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:50.019259 18436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:50.020301 18436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:50.020373 18436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.020447 18436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:50.020486 18436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.030635 18436 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:44677
I20260811 22:34:50.030665 18674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:44677 every 8 connection(s)
I20260811 22:34:50.031598 18436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:50.037071 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 18436
I20260811 22:34:50.037204 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:34:50.037623 19278 external_mini_cluster.cc:1428] Setting key 5b06281fffeb553900085a59196a39f3
I20260811 22:34:50.039716 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33707
--builtin_ntp_servers=127.18.211.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:34:50.041836 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.033549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:57935 (local address 127.18.211.190:33707)
0811 22:34:50.033726 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.033730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.034844 (+ 1114us) server_negotiation.cc:407] Connection header received
0811 22:34:50.036247 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.036249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.036297 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.036373 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:50.037582 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.038187 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.039539 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.039831 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.039930 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.041324 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:50.041332 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:50.041334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:50.041348 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:50.041366 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:50.041373 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:50.041449 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.041623 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.041686 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:34:50.042719 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.033984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33707 (local address 127.18.211.129:57935)
0811 22:34:50.034696 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.034738 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.036053 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.036540 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.036548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.036630 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:50.037424 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.037450 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.038330 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.038333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.039394 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.039406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.040086 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.040089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.040287 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.040991 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:50.040992 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:50.041034 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:50.041099 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:50.041454 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:50.041465 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:50.041472 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.041605 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.041918 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":125,"threads_started":1}
I20260811 22:34:50.044061 18675 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33707
I20260811 22:34:50.044308 18675 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:50.044832 18675 heartbeater.cc:511] Master 127.18.211.190:33707 requested a full tablet report, sending...
I20260811 22:34:50.046366 18333 ts_manager.cc:194] Registered new tserver with Master: a26f75a6027046a381e5475366bbd32a (127.18.211.129:44677)
I20260811 22:34:50.047506 18333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.129:57935
W20260811 22:34:50.161388 18679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:50.161645 18679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:50.161715 18679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:50.165637 18679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:50.165685 18679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:50.165776 18679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:50.170524 18679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:50.171631 18679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:50.173053 18679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:50.182888 18691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:50.182909 18687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:50.182888 18688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:50.183126 18679 server_base.cc:1034] running on GCE node
I20260811 22:34:50.183558 18679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:50.184211 18679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:50.185353 18679 hybrid_clock.cc:648] HybridClock initialized: now 1786487690185319 us; error 46 us; skew 500 ppm
I20260811 22:34:50.187191 18679 webserver.cc:460] Webserver started at http://127.18.211.130:45155/ using document root <none> and password file <none>
I20260811 22:34:50.187763 18679 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:50.187827 18679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:50.188133 18679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:50.190305 18679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "cacaae91d59e40d8a11fd72919eea495"
format_stamp: "Formatted at 2026-08-11 22:34:50 on dist-test-slave-8hhm"
server_key: "d6c75f07895731c9affd6f340f2eb338"
server_key_iv: "6bcfdaac42207c3a4de517ab209543ad"
server_key_version: "encryptionkey@0"
I20260811 22:34:50.190888 18679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "cacaae91d59e40d8a11fd72919eea495"
format_stamp: "Formatted at 2026-08-11 22:34:50 on dist-test-slave-8hhm"
server_key: "d6c75f07895731c9affd6f340f2eb338"
server_key_iv: "6bcfdaac42207c3a4de517ab209543ad"
server_key_version: "encryptionkey@0"
I20260811 22:34:50.194638 18679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:34:50.197113 18700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.198352 18679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:34:50.198473 18679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "cacaae91d59e40d8a11fd72919eea495"
format_stamp: "Formatted at 2026-08-11 22:34:50 on dist-test-slave-8hhm"
server_key: "d6c75f07895731c9affd6f340f2eb338"
server_key_iv: "6bcfdaac42207c3a4de517ab209543ad"
server_key_version: "encryptionkey@0"
I20260811 22:34:50.198575 18679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:50.210261 18679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:50.210963 18679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:50.211159 18679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:50.211750 18679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:50.212711 18679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:50.212759 18679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.212823 18679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:50.212860 18679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.221372 18679 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:42381
I20260811 22:34:50.221426 18892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:42381 every 8 connection(s)
I20260811 22:34:50.222314 18679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:34:50.228256 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 18679
I20260811 22:34:50.228381 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:34:50.228714 19278 external_mini_cluster.cc:1428] Setting key fced752da37d1be385d7451e25049912
I20260811 22:34:50.230707 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33707
--builtin_ntp_servers=127.18.211.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:34:50.232012 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.224224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:59075 (local address 127.18.211.190:33707)
0811 22:34:50.224367 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.224370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.225545 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 22:34:50.226956 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.226959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.227005 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.227082 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:50.227994 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.228809 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.230052 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.230278 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.230394 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.231480 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:50.231489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:50.231492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:50.231504 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:50.231523 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:50.231531 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:50.231611 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.231792 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.231857 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:34:50.232975 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.224624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33707 (local address 127.18.211.130:59075)
0811 22:34:50.225393 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.225439 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.226762 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.227250 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.227257 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.227341 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:50.227846 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.227858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.228934 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.228937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.229916 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.229926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.230437 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.230441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.230615 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.231208 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:50.231210 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:50.231251 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:50.231313 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:50.231626 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:50.231637 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:50.231648 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.231791 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.232071 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":110,"threads_started":1}
I20260811 22:34:50.234187 18893 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33707
I20260811 22:34:50.234443 18893 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:50.234985 18893 heartbeater.cc:511] Master 127.18.211.190:33707 requested a full tablet report, sending...
I20260811 22:34:50.236043 18333 ts_manager.cc:194] Registered new tserver with Master: cacaae91d59e40d8a11fd72919eea495 (127.18.211.130:42381)
I20260811 22:34:50.236660 18333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.130:59075
W20260811 22:34:50.345974 18900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:50.346266 18900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:50.346344 18900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:50.350356 18900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:50.350437 18900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:50.350519 18900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:50.355355 18900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:33707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:50.356527 18900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:50.357990 18900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:50.367846 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:50.367871 18908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:50.367937 18900 server_base.cc:1034] running on GCE node
W20260811 22:34:50.367846 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:50.368594 18900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:50.369206 18900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:50.370424 18900 hybrid_clock.cc:648] HybridClock initialized: now 1786487690370403 us; error 30 us; skew 500 ppm
I20260811 22:34:50.372320 18900 webserver.cc:460] Webserver started at http://127.18.211.131:38323/ using document root <none> and password file <none>
I20260811 22:34:50.372938 18900 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:50.373001 18900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:50.373301 18900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:50.375387 18900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "9639a76df6914dc68a2dded7b4027674"
format_stamp: "Formatted at 2026-08-11 22:34:50 on dist-test-slave-8hhm"
server_key: "1d216542898b530eb3f25e2d851a9fcf"
server_key_iv: "a1f25b6e1227eb19b06418acf1ebd1ac"
server_key_version: "encryptionkey@0"
I20260811 22:34:50.375972 18900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "9639a76df6914dc68a2dded7b4027674"
format_stamp: "Formatted at 2026-08-11 22:34:50 on dist-test-slave-8hhm"
server_key: "1d216542898b530eb3f25e2d851a9fcf"
server_key_iv: "a1f25b6e1227eb19b06418acf1ebd1ac"
server_key_version: "encryptionkey@0"
I20260811 22:34:50.379787 18900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:34:50.382239 18917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.383435 18900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:50.383531 18900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "9639a76df6914dc68a2dded7b4027674"
format_stamp: "Formatted at 2026-08-11 22:34:50 on dist-test-slave-8hhm"
server_key: "1d216542898b530eb3f25e2d851a9fcf"
server_key_iv: "a1f25b6e1227eb19b06418acf1ebd1ac"
server_key_version: "encryptionkey@0"
I20260811 22:34:50.383663 18900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:50.407780 18900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:50.408519 18900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:50.408739 18900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:50.409396 18900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:50.410356 18900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:50.410403 18900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.410444 18900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:50.410460 18900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:50.419529 18900 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:35869
I20260811 22:34:50.419554 19119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:35869 every 8 connection(s)
I20260811 22:34:50.420480 18900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:50.429250 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 18900
I20260811 22:34:50.429404 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:34:50.429773 19278 external_mini_cluster.cc:1428] Setting key 370b4f68a3a1792499d87407af30b5e5
I20260811 22:34:50.430325 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.422472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:52161 (local address 127.18.211.190:33707)
0811 22:34:50.422625 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.422629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.423721 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 22:34:50.425188 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.425192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.425269 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.425375 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:50.426287 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.426813 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.427967 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.428201 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.428319 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.429731 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:50.429735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:50.429738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:50.429754 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:50.429776 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:50.429789 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:50.429883 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.430124 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.430190 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:34:50.431398 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.422876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:33707 (local address 127.18.211.131:52161)
0811 22:34:50.423563 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.423605 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.424967 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.425558 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.425564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.425649 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:50.426139 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.426151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.426942 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.426945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.427837 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.427847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.428382 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.428387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.428557 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.429300 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:50.429301 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:50.429369 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:50.429458 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:50.429935 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:50.429948 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:50.429955 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.430096 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.430422 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":123,"threads_started":1}
I20260811 22:34:50.432425 19120 heartbeater.cc:347] Connected to a master server at 127.18.211.190:33707
I20260811 22:34:50.432659 19120 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:50.433187 19120 heartbeater.cc:511] Master 127.18.211.190:33707 requested a full tablet report, sending...
I20260811 22:34:50.434212 18333 ts_manager.cc:194] Registered new tserver with Master: 9639a76df6914dc68a2dded7b4027674 (127.18.211.131:35869)
I20260811 22:34:50.434827 18333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.131:52161
I20260811 22:34:50.443795 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:34:50.449420 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.445918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.18.211.190:33707)
0811 22:34:50.446060 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.446063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.446198 (+ 135us) server_negotiation.cc:407] Connection header received
0811 22:34:50.446302 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.446304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.446344 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.446413 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:50.446651 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.447172 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.447971 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.448192 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.448278 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.449009 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:50.449012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:50.449014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:50.449025 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:50.449041 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:50.449048 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:50.449107 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.449200 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.449312 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:34:50.452193 18333 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:34:50.455302 18333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:34:50.469933 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.463743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:42381 (local address 127.0.0.1:34888)
0811 22:34:50.464240 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.464290 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.464442 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.465389 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.465396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.465424 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:50.465634 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.465643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.467117 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.467121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.468267 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.468277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.469326 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.469332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.469451 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.469489 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.469626 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.469751 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":20,"thread_start_us":66,"threads_started":1}
I20260811 22:34:50.469970 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.463913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.18.211.130:42381)
0811 22:34:50.464772 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.464786 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.464811 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:34:50.464892 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.464903 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.465172 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.465503 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:50.465759 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.466967 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.468403 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.469172 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.469332 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.469515 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.469679 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.469790 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":148,"threads_started":1}
I20260811 22:34:50.470369 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.464552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.18.211.129:44677)
0811 22:34:50.464922 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.464938 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.464964 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:34:50.465049 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.465057 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.465286 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.465459 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:50.465987 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.467114 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.468550 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.469278 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.469398 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.469580 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.470026 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.470221 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":136,"threads_started":1}
I20260811 22:34:50.470491 19145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.464439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:35869 (local address 127.0.0.1:33822)
0811 22:34:50.465873 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.465886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.465974 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.466418 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.466421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.466437 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:50.466522 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.466530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.467681 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.467684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.468819 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.468829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.470089 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.470093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.470183 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.470201 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.470292 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.470359 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1368,"mutex_wait_us":26,"thread_start_us":91,"threads_started":1}
I20260811 22:34:50.470585 19152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.464901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.18.211.131:35869)
0811 22:34:50.465323 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.465338 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.466070 (+ 732us) server_negotiation.cc:407] Connection header received
0811 22:34:50.466137 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.466140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.466273 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.466422 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:50.466632 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.467553 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.468941 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.469753 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.469865 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.470021 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.470327 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.470417 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":155,"threads_started":1}
I20260811 22:34:50.470816 19142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.463939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44677 (local address 127.0.0.1:59726)
0811 22:34:50.464359 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.464374 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.464487 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.465415 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.465417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.465428 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:50.465858 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.465864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.467228 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.467232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.468407 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.468418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.469649 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.469652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.469743 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.469762 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.469830 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.469882 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":27,"thread_start_us":97,"threads_started":1}
I20260811 22:34:50.473492 19002 tablet_service.cc:1467] Processing CreateTablet for tablet e36847c0d05b4bc0aecfb56d38b06ab1 (DEFAULT_TABLE table=test-table [id=ec0f8f8f802a48f084fc2b48910da81f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:50.473562 18775 tablet_service.cc:1467] Processing CreateTablet for tablet e36847c0d05b4bc0aecfb56d38b06ab1 (DEFAULT_TABLE table=test-table [id=ec0f8f8f802a48f084fc2b48910da81f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:50.473912 18556 tablet_service.cc:1467] Processing CreateTablet for tablet e36847c0d05b4bc0aecfb56d38b06ab1 (DEFAULT_TABLE table=test-table [id=ec0f8f8f802a48f084fc2b48910da81f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:50.474611 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e36847c0d05b4bc0aecfb56d38b06ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:50.474617 18775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e36847c0d05b4bc0aecfb56d38b06ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:50.474797 18556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e36847c0d05b4bc0aecfb56d38b06ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:50.481077 19154 tablet_bootstrap.cc:492] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Bootstrap starting.
I20260811 22:34:50.482362 19160 tablet_bootstrap.cc:492] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Bootstrap starting.
I20260811 22:34:50.483618 19154 tablet_bootstrap.cc:654] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:50.484076 19160 tablet_bootstrap.cc:654] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:50.484423 19154 log.cc:824] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:50.484805 19160 log.cc:824] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:50.485129 19155 tablet_bootstrap.cc:492] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: Bootstrap starting.
I20260811 22:34:50.486680 19154 tablet_bootstrap.cc:492] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: No bootstrap required, opened a new log
I20260811 22:34:50.486881 19154 ts_tablet_manager.cc:1397] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:34:50.487020 19160 tablet_bootstrap.cc:492] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: No bootstrap required, opened a new log
I20260811 22:34:50.487242 19160 ts_tablet_manager.cc:1397] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:34:50.487355 19155 tablet_bootstrap.cc:654] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:50.488204 19155 log.cc:824] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:50.490381 19155 tablet_bootstrap.cc:492] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: No bootstrap required, opened a new log
I20260811 22:34:50.490576 19155 ts_tablet_manager.cc:1397] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:34:50.490926 19160 raft_consensus.cc:348] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.491176 19160 raft_consensus.cc:374] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:50.491258 19160 raft_consensus.cc:729] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacaae91d59e40d8a11fd72919eea495, State: Initialized, Role: FOLLOWER
I20260811 22:34:50.491602 19154 raft_consensus.cc:348] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.491870 19154 raft_consensus.cc:374] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:50.491784 19160 consensus_queue.cc:260] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.491967 19154 raft_consensus.cc:729] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9639a76df6914dc68a2dded7b4027674, State: Initialized, Role: FOLLOWER
I20260811 22:34:50.492422 19154 consensus_queue.cc:260] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.492830 19160 ts_tablet_manager.cc:1428] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:34:50.493240 18893 heartbeater.cc:503] Master 127.18.211.190:33707 was elected leader, sending a full tablet report...
I20260811 22:34:50.493618 19154 ts_tablet_manager.cc:1428] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:34:50.494001 19120 heartbeater.cc:503] Master 127.18.211.190:33707 was elected leader, sending a full tablet report...
I20260811 22:34:50.495105 19155 raft_consensus.cc:348] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.495373 19155 raft_consensus.cc:374] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:50.495438 19155 raft_consensus.cc:729] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26f75a6027046a381e5475366bbd32a, State: Initialized, Role: FOLLOWER
I20260811 22:34:50.495882 19155 consensus_queue.cc:260] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.496955 19155 ts_tablet_manager.cc:1428] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:34:50.497357 18675 heartbeater.cc:503] Master 127.18.211.190:33707 was elected leader, sending a full tablet report...
W20260811 22:34:50.532804 18676 tablet.cc:2369] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:34:50.671547 19122 tablet.cc:2369] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:34:50.704243 19165 raft_consensus.cc:482] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:50.704491 19165 raft_consensus.cc:504] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.705839 19165 leader_election.cc:288] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a26f75a6027046a381e5475366bbd32a (127.18.211.129:44677), 9639a76df6914dc68a2dded7b4027674 (127.18.211.131:35869)
I20260811 22:34:50.709760 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.706034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44677 (local address 127.18.211.130:58195)
0811 22:34:50.706258 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.706275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.706377 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.706691 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.706694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.706716 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:50.706846 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.706857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.707736 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.707739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.708623 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.708633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.709329 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.709336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.709432 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.709461 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.709539 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.709608 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 22:34:50.709780 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.706169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:58195 (local address 127.18.211.129:44677)
0811 22:34:50.706319 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.706324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.706343 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:34:50.706512 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.706516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.706583 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.706665 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:50.706978 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.707610 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.708761 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.709180 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.709342 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.709416 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.709576 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.709638 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:34:50.709904 19152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.706285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:50919 (local address 127.18.211.131:35869)
0811 22:34:50.706420 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.706426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.706656 (+ 230us) server_negotiation.cc:407] Connection header received
0811 22:34:50.706748 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.706751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.706798 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.706890 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:50.707209 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.707779 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.708824 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.709265 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.709377 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.709429 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.709715 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.709783 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:34:50.709919 19170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.706163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:35869 (local address 127.18.211.130:50919)
0811 22:34:50.706539 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.706553 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:50.706659 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:50.706922 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:50.706926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:50.706946 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:50.707090 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.707096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.707894 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.707897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.708671 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:50.708701 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.709496 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.709502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:50.709591 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.709610 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:34:50.709680 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:34:50.709743 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":99,"threads_started":1}
I20260811 22:34:50.710500 18591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36847c0d05b4bc0aecfb56d38b06ab1" candidate_uuid: "cacaae91d59e40d8a11fd72919eea495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a26f75a6027046a381e5475366bbd32a" is_pre_election: true
I20260811 22:34:50.710798 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36847c0d05b4bc0aecfb56d38b06ab1" candidate_uuid: "cacaae91d59e40d8a11fd72919eea495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9639a76df6914dc68a2dded7b4027674" is_pre_election: true
I20260811 22:34:50.710835 18591 raft_consensus.cc:2454] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cacaae91d59e40d8a11fd72919eea495 in term 0.
I20260811 22:34:50.711082 19036 raft_consensus.cc:2454] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cacaae91d59e40d8a11fd72919eea495 in term 0.
I20260811 22:34:50.711365 18705 leader_election.cc:302] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a26f75a6027046a381e5475366bbd32a, cacaae91d59e40d8a11fd72919eea495; no voters:
I20260811 22:34:50.711709 19165 raft_consensus.cc:2781] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:34:50.711791 19165 raft_consensus.cc:482] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:34:50.711839 19165 raft_consensus.cc:3037] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:50.712910 19165 raft_consensus.cc:504] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.713318 19165 leader_election.cc:288] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [CANDIDATE]: Term 1 election: Requested vote from peers a26f75a6027046a381e5475366bbd32a (127.18.211.129:44677), 9639a76df6914dc68a2dded7b4027674 (127.18.211.131:35869)
I20260811 22:34:50.713670 19036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36847c0d05b4bc0aecfb56d38b06ab1" candidate_uuid: "cacaae91d59e40d8a11fd72919eea495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9639a76df6914dc68a2dded7b4027674"
I20260811 22:34:50.713680 18591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36847c0d05b4bc0aecfb56d38b06ab1" candidate_uuid: "cacaae91d59e40d8a11fd72919eea495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a26f75a6027046a381e5475366bbd32a"
I20260811 22:34:50.713785 19036 raft_consensus.cc:3037] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:50.713816 18591 raft_consensus.cc:3037] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:50.714883 18591 raft_consensus.cc:2454] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cacaae91d59e40d8a11fd72919eea495 in term 1.
I20260811 22:34:50.714884 19036 raft_consensus.cc:2454] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cacaae91d59e40d8a11fd72919eea495 in term 1.
I20260811 22:34:50.715217 18706 leader_election.cc:302] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9639a76df6914dc68a2dded7b4027674, cacaae91d59e40d8a11fd72919eea495; no voters:
I20260811 22:34:50.715401 19165 raft_consensus.cc:2781] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:50.715634 19165 raft_consensus.cc:686] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 1 LEADER]: Becoming Leader. State: Replica: cacaae91d59e40d8a11fd72919eea495, State: Running, Role: LEADER
I20260811 22:34:50.716032 19165 consensus_queue.cc:237] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } }
I20260811 22:34:50.719871 18333 catalog_manager.cc:5495] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 reported cstate change: term changed from 0 to 1, leader changed from <none> to cacaae91d59e40d8a11fd72919eea495 (127.18.211.130). New cstate: current_term: 1 leader_uuid: "cacaae91d59e40d8a11fd72919eea495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cacaae91d59e40d8a11fd72919eea495" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 } health_report { overall_health: UNKNOWN } } }
W20260811 22:34:50.723439 18898 tablet.cc:2369] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:34:50.762791 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.759290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.18.211.130:42381)
0811 22:34:50.759428 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:50.759432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:50.759446 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:34:50.759566 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:50.759569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:50.759628 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:50.759719 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:50.759938 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.760429 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.761204 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:50.761451 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:50.761540 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:50.761847 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:50.762346 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:34:50.762452 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:50.762507 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 22:34:50.762554 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:34:50.769611 19036 raft_consensus.cc:1261] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 1 FOLLOWER]: Refusing update from remote peer cacaae91d59e40d8a11fd72919eea495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:50.769641 18591 raft_consensus.cc:1261] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 1 FOLLOWER]: Refusing update from remote peer cacaae91d59e40d8a11fd72919eea495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:50.770390 19165 consensus_queue.cc:1035] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a26f75a6027046a381e5475366bbd32a" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 44677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:50.770613 19172 consensus_queue.cc:1035] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9639a76df6914dc68a2dded7b4027674" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:50.782518 19181 mvcc.cc:205] Tried to move back new op lower bound from 7317453581384224768 to 7317453581176643584. Current Snapshot: MvccSnapshot[applied={T|T < 7317453581384224768}]
I20260811 22:34:50.789577 18333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:50.789757 18333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:50.792424 18333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71243adbef1f4b9c9c423c274064c48f: Sending DeleteTablet for 3 replicas of tablet e36847c0d05b4bc0aecfb56d38b06ab1
I20260811 22:34:50.793188 18556 tablet_service.cc:1514] Processing DeleteTablet for tablet e36847c0d05b4bc0aecfb56d38b06ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:50 UTC) from {username='slave'} at 127.0.0.1:59726
I20260811 22:34:50.793726 18775 tablet_service.cc:1514] Processing DeleteTablet for tablet e36847c0d05b4bc0aecfb56d38b06ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:50 UTC) from {username='slave'} at 127.0.0.1:34888
I20260811 22:34:50.793795 19198 tablet_replica.cc:330] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: stopping tablet replica
I20260811 22:34:50.794140 19198 raft_consensus.cc:2229] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:50.794286 19199 tablet_replica.cc:330] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: stopping tablet replica
I20260811 22:34:50.794482 19198 raft_consensus.cc:2258] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:50.794652 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet e36847c0d05b4bc0aecfb56d38b06ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:50 UTC) from {username='slave'} at 127.0.0.1:33822
I20260811 22:34:50.794895 19199 raft_consensus.cc:2229] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:34:50.795353 19199 raft_consensus.cc:2258] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:50.796084 19198 ts_tablet_manager.cc:1905] T e36847c0d05b4bc0aecfb56d38b06ab1 P a26f75a6027046a381e5475366bbd32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:50.796160 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 18436
I20260811 22:34:50.797242 19199 ts_tablet_manager.cc:1905] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:50.800601 19203 tablet_replica.cc:330] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: stopping tablet replica
I20260811 22:34:50.800963 19203 raft_consensus.cc:2229] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:50.801083 19199 ts_tablet_manager.cc:1918] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:50.801226 19199 log.cc:1196] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/e36847c0d05b4bc0aecfb56d38b06ab1
I20260811 22:34:50.801584 19199 ts_tablet_manager.cc:1939] T e36847c0d05b4bc0aecfb56d38b06ab1 P cacaae91d59e40d8a11fd72919eea495: Deleting consensus metadata
I20260811 22:34:50.801584 19203 raft_consensus.cc:2258] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:50.803164 19203 ts_tablet_manager.cc:1905] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:50.805758 18309 catalog_manager.cc:4841] TS cacaae91d59e40d8a11fd72919eea495 (127.18.211.130:42381): tablet e36847c0d05b4bc0aecfb56d38b06ab1 (table test-table [id=ec0f8f8f802a48f084fc2b48910da81f]) successfully deleted
I20260811 22:34:50.805809 19203 ts_tablet_manager.cc:1918] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:50.805896 19203 log.cc:1196] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/e36847c0d05b4bc0aecfb56d38b06ab1
I20260811 22:34:50.806195 19203 ts_tablet_manager.cc:1939] T e36847c0d05b4bc0aecfb56d38b06ab1 P 9639a76df6914dc68a2dded7b4027674: Deleting consensus metadata
I20260811 22:34:50.807380 18309 catalog_manager.cc:4841] TS 9639a76df6914dc68a2dded7b4027674 (127.18.211.131:35869): tablet e36847c0d05b4bc0aecfb56d38b06ab1 (table test-table [id=ec0f8f8f802a48f084fc2b48910da81f]) successfully deleted
W20260811 22:34:50.810236 18307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:44677 (error 108)
I20260811 22:34:50.811160 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 18679
I20260811 22:34:50.811403 19142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:50.811037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:44677 (local address 127.0.0.1:59742)
0811 22:34:50.811205 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:50.811296 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:34:50.821048 18307 catalog_manager.cc:4584] TS a26f75a6027046a381e5475366bbd32a (127.18.211.129:44677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e36847c0d05b4bc0aecfb56d38b06ab1: Network error: Client connection negotiation failed: client connection to 127.18.211.129:44677: connect: Connection refused (error 111)
I20260811 22:34:50.826990 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 18900
I20260811 22:34:50.853654 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 18281
2026-08-11T22:34:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1294 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:34:50 dist-test-slave-8hhm krb5kdc[19210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:34:50 dist-test-slave-8hhm krb5kdc[19210](info): set up 2 sockets
Aug 11 22:34:50 dist-test-slave-8hhm krb5kdc[19210](info): commencing operation
krb5kdc: starting...
W20260811 22:34:52.161073 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:34:52 dist-test-slave-8hhm krb5kdc[19210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:34:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:34:52.297204 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:35003
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:35003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:52.411191 19227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:52.411507 19227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:52.411582 19227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:52.415565 19227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:52.415657 19227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:52.415693 19227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:52.415730 19227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:52.415766 19227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:52.420935 19227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:35003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:35003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:52.421994 19227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:52.423018 19227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:52.431600 19240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:52.431607 19239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:52.431833 19227 server_base.cc:1034] running on GCE node
W20260811 22:34:52.431617 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:52.432528 19227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:52.433702 19227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:52.434881 19227 hybrid_clock.cc:648] HybridClock initialized: now 1786487692434871 us; error 27 us; skew 500 ppm
Aug 11 22:34:52 dist-test-slave-8hhm krb5kdc[19210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:52.437677 19227 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:34:52.438547 19227 webserver.cc:460] Webserver started at http://127.18.211.190:46271/ using document root <none> and password file <none>
I20260811 22:34:52.439093 19227 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:52.439153 19227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:52.439327 19227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:52.441448 19227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "3c5a0dba9fe340fe9b2435b89f861521"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "ac8a3b6f0f19df01b71ee7895c854d93"
server_key_iv: "c57b840afd9d7cee6d74e45ba4befd5d"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.441989 19227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "3c5a0dba9fe340fe9b2435b89f861521"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "ac8a3b6f0f19df01b71ee7895c854d93"
server_key_iv: "c57b840afd9d7cee6d74e45ba4befd5d"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.445772 19227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:34:52.448143 19256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:52.449445 19227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:52.449569 19227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "3c5a0dba9fe340fe9b2435b89f861521"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "ac8a3b6f0f19df01b71ee7895c854d93"
server_key_iv: "c57b840afd9d7cee6d74e45ba4befd5d"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.449687 19227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:52.464847 19227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:52.467855 19227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:52.468070 19227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:52.475582 19227 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:35003
I20260811 22:34:52.475612 19371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:35003 every 8 connection(s)
I20260811 22:34:52.476569 19227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:52.479527 19372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:52.484678 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19227
I20260811 22:34:52.484799 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:34:52.485133 19278 external_mini_cluster.cc:1428] Setting key 86a011452533f52b9d34cda376af67b9
I20260811 22:34:52.485483 19372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521: Bootstrap starting.
I20260811 22:34:52.488313 19372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:52.489279 19372 log.cc:824] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:52.491947 19372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521: No bootstrap required, opened a new log
Aug 11 22:34:52 dist-test-slave-8hhm krb5kdc[19210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:52.496325 19372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5a0dba9fe340fe9b2435b89f861521" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35003 } }
I20260811 22:34:52.496654 19372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:52.496714 19372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5a0dba9fe340fe9b2435b89f861521, State: Initialized, Role: FOLLOWER
I20260811 22:34:52.497294 19372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5a0dba9fe340fe9b2435b89f861521" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35003 } }
I20260811 22:34:52.497434 19372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:52.497519 19372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:52.497638 19372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:52.498571 19372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5a0dba9fe340fe9b2435b89f861521" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35003 } }
I20260811 22:34:52.498936 19372 leader_election.cc:302] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c5a0dba9fe340fe9b2435b89f861521; no voters:
I20260811 22:34:52.499317 19372 leader_election.cc:288] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:34:52.499363 19384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:52.499579 19384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [term 1 LEADER]: Becoming Leader. State: Replica: 3c5a0dba9fe340fe9b2435b89f861521, State: Running, Role: LEADER
I20260811 22:34:52.499900 19384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5a0dba9fe340fe9b2435b89f861521" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35003 } }
I20260811 22:34:52.500669 19372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:52.501587 19388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c5a0dba9fe340fe9b2435b89f861521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5a0dba9fe340fe9b2435b89f861521" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35003 } } }
I20260811 22:34:52.501711 19388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:52.501992 19385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c5a0dba9fe340fe9b2435b89f861521. Latest consensus state: current_term: 1 leader_uuid: "3c5a0dba9fe340fe9b2435b89f861521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5a0dba9fe340fe9b2435b89f861521" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35003 } } }
I20260811 22:34:52.502146 19385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:52.502648 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:52.486641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.18.211.190:35003)
0811 22:34:52.487086 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:52.487107 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:52.487183 (+ 76us) server_negotiation.cc:407] Connection header received
0811 22:34:52.488114 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:52.488141 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:52.488611 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:52.489013 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:52.490422 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:52.491906 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:52.492930 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:52.493713 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:52.493891 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:52.496499 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:52.496505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:52.496523 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:52.496557 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:52.498565 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:52.499042 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:52.499048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:52.499054 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:52.499187 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:52.499522 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:52.499525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:52.499526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:52.500092 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:52.500275 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:52.500693 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 22:34:52.501099 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":125,"threads_started":1}
I20260811 22:34:52.503562 19396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:52.506358 19396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:52.511950 19396 catalog_manager.cc:1347] Generated new cluster ID: 73b6c70d98db4c04a74be2fb6de3c9f2
I20260811 22:34:52.512015 19396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:52.527820 19396 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:34:52.529107 19396 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:52.537685 19396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521: Generated new TSK 0
I20260811 22:34:52.538419 19396 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:34:52.574628 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:35003
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:52.690536 19420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:52.690851 19420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:52.690934 19420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:52.695057 19420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:52.695144 19420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:52.695236 19420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:52.700434 19420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:35003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:52.701752 19420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:52.702962 19420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:52.713245 19433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:52.713445 19420 server_base.cc:1034] running on GCE node
W20260811 22:34:52.713274 19429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:52.713276 19435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:52.713989 19420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:52.714635 19420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:52.715811 19420 hybrid_clock.cc:648] HybridClock initialized: now 1786487692715795 us; error 27 us; skew 500 ppm
Aug 11 22:34:52 dist-test-slave-8hhm krb5kdc[19210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:52.718833 19420 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:34:52.719988 19420 webserver.cc:460] Webserver started at http://127.18.211.129:44449/ using document root <none> and password file <none>
I20260811 22:34:52.720644 19420 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:52.720714 19420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:52.720898 19420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:52.723052 19420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "c9744862068344e8a6530b1753ede998"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "9ac55a43d818d19df78a4af1f539925e"
server_key_iv: "716ab11ad2a6fdf5938c87c7ac7c7c52"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.723592 19420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "c9744862068344e8a6530b1753ede998"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "9ac55a43d818d19df78a4af1f539925e"
server_key_iv: "716ab11ad2a6fdf5938c87c7ac7c7c52"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.727454 19420 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:34:52.729913 19444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:52.731201 19420 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:52.731343 19420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "c9744862068344e8a6530b1753ede998"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "9ac55a43d818d19df78a4af1f539925e"
server_key_iv: "716ab11ad2a6fdf5938c87c7ac7c7c52"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.731458 19420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:52.749701 19420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:52.753557 19420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:52.753780 19420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:52.754419 19420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:52.755374 19420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:52.755437 19420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:52.755550 19420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:52.755589 19420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:52.765372 19420 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:32929
I20260811 22:34:52.765446 19633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:32929 every 8 connection(s)
I20260811 22:34:52.766330 19420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:52.772212 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19420
I20260811 22:34:52.772352 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:34:52.772666 19278 external_mini_cluster.cc:1428] Setting key b0ef7069f232fbb7dda060dbdf13b874
Aug 11 22:34:52 dist-test-slave-8hhm krb5kdc[19210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:52.781769 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:52.768229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:47095 (local address 127.18.211.190:35003)
0811 22:34:52.768407 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:52.768412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:52.769260 (+ 848us) server_negotiation.cc:407] Connection header received
0811 22:34:52.770626 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:52.770628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:52.770677 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:52.770750 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:52.771929 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:52.772540 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:52.773909 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:52.774265 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:52.774350 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:52.777611 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:52.777614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:52.777616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:52.777639 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:52.779418 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:52.779955 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:52.779962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:52.779966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:52.780072 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:52.780409 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:52.780416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:52.780420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:52.780684 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:52.780808 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:52.781349 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 22:34:52.781506 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:34:52.782464 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:52.768596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:35003 (local address 127.18.211.129:47095)
0811 22:34:52.769071 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:52.769115 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:52.770410 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:52.770913 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:52.770920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:52.771299 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:52.771796 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:52.771810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:52.772670 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:52.772674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:52.773743 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:52.773754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:52.774430 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:52.774437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:52.774652 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:52.775274 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:52.775300 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:52.777420 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:52.779543 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:52.779548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:52.779560 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:52.779824 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:52.780167 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:52.780170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:52.780171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:52.780301 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:52.780806 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:52.780811 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:52.781046 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 22:34:52.781204 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 22:34:52.781519 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260811 22:34:52.783778 19634 heartbeater.cc:347] Connected to a master server at 127.18.211.190:35003
I20260811 22:34:52.784016 19634 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:52.784499 19634 heartbeater.cc:511] Master 127.18.211.190:35003 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:34:52.786111 19299 ts_manager.cc:194] Registered new tserver with Master: c9744862068344e8a6530b1753ede998 (127.18.211.129:32929)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:34:52.787457 19299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:47095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:34:52.804476 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:35003
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:52.924475 19641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:52.924741 19641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:52.924785 19641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:52.928689 19641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:52.928741 19641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:52.928797 19641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:52.934221 19641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:35003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:52.935374 19641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:52.936465 19641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:52.946475 19651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:52.946487 19652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:52.946463 19654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:52.947062 19641 server_base.cc:1034] running on GCE node
I20260811 22:34:52.947489 19641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:52.948133 19641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:52.949290 19641 hybrid_clock.cc:648] HybridClock initialized: now 1786487692949263 us; error 31 us; skew 500 ppm
Aug 11 22:34:52 dist-test-slave-8hhm krb5kdc[19210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:52.952350 19641 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:34:52.953347 19641 webserver.cc:460] Webserver started at http://127.18.211.130:36561/ using document root <none> and password file <none>
I20260811 22:34:52.953924 19641 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:52.954025 19641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:52.954259 19641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:52.956420 19641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "ddf9176ed915155837f10d57fdc04862"
server_key_iv: "df388cf5868ecd3c93374c4d37181db1"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.957015 19641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "ddf9176ed915155837f10d57fdc04862"
server_key_iv: "df388cf5868ecd3c93374c4d37181db1"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.961056 19641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:34:52.963702 19665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:52.965041 19641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:52.965260 19641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8"
format_stamp: "Formatted at 2026-08-11 22:34:52 on dist-test-slave-8hhm"
server_key: "ddf9176ed915155837f10d57fdc04862"
server_key_iv: "df388cf5868ecd3c93374c4d37181db1"
server_key_version: "encryptionkey@0"
I20260811 22:34:52.965396 19641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:52.983194 19641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:52.986788 19641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:52.987020 19641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:52.987658 19641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:52.988570 19641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:52.988638 19641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:52.988714 19641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:52.988757 19641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:52.997665 19641 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:33157
I20260811 22:34:52.997752 19856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:33157 every 8 connection(s)
I20260811 22:34:52.998613 19641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:34:53.002141 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19641
I20260811 22:34:53.002310 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:34:53.002611 19278 external_mini_cluster.cc:1428] Setting key f7d33d44f33f3f721ddb277dd7ea6248
Aug 11 22:34:53 dist-test-slave-8hhm krb5kdc[19210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:34:53.013685 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.000538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:50735 (local address 127.18.211.190:35003)
0811 22:34:53.000708 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.000711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.001718 (+ 1007us) server_negotiation.cc:407] Connection header received
0811 22:34:53.003324 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.003327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.003389 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.003469 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:53.004797 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.005402 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.006586 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.006862 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.006980 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.009901 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:53.009905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:53.009908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:53.009942 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:53.011453 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:53.012007 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:53.012011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:53.012012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:53.012091 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:53.012440 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:53.012444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:53.012446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:53.012633 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:53.012759 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.013372 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.013508 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:34:53.014520 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.000930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:35003 (local address 127.18.211.130:50735)
0811 22:34:53.001544 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.001591 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:53.003063 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:53.003653 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:53.003660 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:53.004127 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:53.004646 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.004660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.005537 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.005540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.006454 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.006463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.007028 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.007032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.007213 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.007862 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:53.007879 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:53.009725 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:53.011581 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:53.011587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:53.011604 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:53.011890 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:53.012183 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:53.012187 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:53.012189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:53.012342 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:53.012760 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:53.012768 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:53.013077 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 22:34:53.013323 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 22:34:53.013659 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":174,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:34:53.015738 19858 heartbeater.cc:347] Connected to a master server at 127.18.211.190:35003
I20260811 22:34:53.015969 19858 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:53.016543 19858 heartbeater.cc:511] Master 127.18.211.190:35003 requested a full tablet report, sending...
I20260811 22:34:53.017789 19299 ts_manager.cc:194] Registered new tserver with Master: d245ada7bb8a40b1af8a75c81bf5a1d8 (127.18.211.130:33157)
I20260811 22:34:53.018638 19299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:50735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:34:53.033085 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:35003
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:53.146556 19866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:53.146808 19866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:53.146852 19866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:53.150830 19866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:53.150885 19866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:53.150940 19866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:53.156188 19866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:35003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:53.157143 19866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:53.158195 19866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:53.167963 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:53.167928 19885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:53.168016 19876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:53.168056 19866 server_base.cc:1034] running on GCE node
I20260811 22:34:53.168612 19866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:53.169327 19866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:53.170503 19866 hybrid_clock.cc:648] HybridClock initialized: now 1786487693170477 us; error 35 us; skew 500 ppm
Aug 11 22:34:53 dist-test-slave-8hhm krb5kdc[19210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487693, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:53.173266 19866 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:34:53.174106 19866 webserver.cc:460] Webserver started at http://127.18.211.131:46017/ using document root <none> and password file <none>
I20260811 22:34:53.174606 19866 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:53.174666 19866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:53.174839 19866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:53.177188 19866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "5de6aa256d584aa48bd4fac41a182855"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "10d4982a0dfe62c348c75310c70ab9b6"
server_key_iv: "6cf6d2383458f5996523124ee728fc61"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.177790 19866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "5de6aa256d584aa48bd4fac41a182855"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "10d4982a0dfe62c348c75310c70ab9b6"
server_key_iv: "6cf6d2383458f5996523124ee728fc61"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.181414 19866 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:34:53.184180 19898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:53.185830 19866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:34:53.185928 19866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "5de6aa256d584aa48bd4fac41a182855"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "10d4982a0dfe62c348c75310c70ab9b6"
server_key_iv: "6cf6d2383458f5996523124ee728fc61"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.186036 19866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:53.223840 19866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:53.226734 19866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:53.226889 19866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:53.227424 19866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:53.228216 19866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:53.228266 19866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:53.228308 19866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:53.228329 19866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:53.237546 19866 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:35571
I20260811 22:34:53.237574 20096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:35571 every 8 connection(s)
I20260811 22:34:53.238425 19866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:53.241037 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 19866
I20260811 22:34:53.241228 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:34:53.241606 19278 external_mini_cluster.cc:1428] Setting key 3afeb20027d448e962ed793aed20939c
Aug 11 22:34:53 dist-test-slave-8hhm krb5kdc[19210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487693, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:53.252506 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.240332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:53963 (local address 127.18.211.190:35003)
0811 22:34:53.240465 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.240469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.241476 (+ 1007us) server_negotiation.cc:407] Connection header received
0811 22:34:53.242975 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.242979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.243036 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.243136 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:53.244402 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.244913 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.246117 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.246346 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.246457 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.248961 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:53.248964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:53.248966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:53.248990 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:53.250454 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:53.250975 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:53.250979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:53.250981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:53.251062 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:53.251398 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:53.251402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:53.251404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:53.251582 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:53.251685 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.252219 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.252338 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:34:53.253407 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.240747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:35003 (local address 127.18.211.131:53963)
0811 22:34:53.241303 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.241363 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:53.242752 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:53.243312 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:53.243319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:53.243746 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:53.244266 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.244279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.245041 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.245044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.245990 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.246000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.246497 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.246501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.246669 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.247264 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:53.247282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:53.248795 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:53.250576 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:53.250582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:53.250598 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:53.250868 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:53.251153 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:53.251156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:53.251159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:53.251300 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:53.251716 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:53.251723 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:53.251966 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 22:34:53.252162 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:34:53.252484 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":129,"threads_started":1}
I20260811 22:34:53.254534 20099 heartbeater.cc:347] Connected to a master server at 127.18.211.190:35003
I20260811 22:34:53.254778 20099 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:53.255385 20099 heartbeater.cc:511] Master 127.18.211.190:35003 requested a full tablet report, sending...
I20260811 22:34:53.256373 19299 ts_manager.cc:194] Registered new tserver with Master: 5de6aa256d584aa48bd4fac41a182855 (127.18.211.131:35571)
I20260811 22:34:53.256924 19299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:53963
I20260811 22:34:53.267091 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:34:53.277012 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.269165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.18.211.190:35003)
0811 22:34:53.269328 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.269333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.269508 (+ 175us) server_negotiation.cc:407] Connection header received
0811 22:34:53.269619 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.269621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.269659 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.269728 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:53.270502 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.270996 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.271863 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.272068 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.272144 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.273865 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:53.273868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:53.273870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:53.273892 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:53.275328 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:53.275827 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:53.275829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:53.275830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:53.275903 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:53.276184 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:53.276186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:53.276187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:53.276376 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:53.276491 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.276733 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.276838 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:34:53.280061 19299 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:34:53.283370 19299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:34:53.298771 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.292171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.18.211.131:35571)
0811 22:34:53.293711 (+ 1540us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.293726 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.293750 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:34:53.293856 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.293862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.294077 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.294211 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:53.294539 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.295441 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.297003 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.298011 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.298157 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.298317 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.298506 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.298611 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1426,"thread_start_us":177,"threads_started":1}
I20260811 22:34:53.298767 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.292049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:35571 (local address 127.0.0.1:52066)
0811 22:34:53.292401 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.292456 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:53.292588 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:53.294245 (+ 1657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:53.294251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:53.294276 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:53.294419 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.294427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.295558 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.295563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.296865 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.296877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.298142 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.298149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.298292 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.298339 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 22:34:53.298459 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:34:53.298586 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260811 22:34:53.299110 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.293361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.18.211.129:32929)
0811 22:34:53.293803 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.293809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.293833 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:34:53.293911 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.293932 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.294124 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.294292 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:53.294581 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.295823 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.297536 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.298202 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.298326 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.298458 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.298843 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.298971 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":101,"threads_started":1}
I20260811 22:34:53.299374 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.293164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:32929 (local address 127.0.0.1:55484)
0811 22:34:53.293426 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.293441 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:53.293536 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:53.294284 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:53.294287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:53.294307 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:53.294421 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.294428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.295946 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.295950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.297054 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.297062 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.298535 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.298539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.298616 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.298636 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:34:53.298704 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:34:53.298762 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":120,"threads_started":1}
I20260811 22:34:53.299966 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.292782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:33157 (local address 127.0.0.1:45768)
0811 22:34:53.293087 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.293105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:53.293200 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:53.295036 (+ 1836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:53.295040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:53.295061 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:53.295149 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.295155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.296646 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.296649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.297983 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.297995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.299221 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.299225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.299631 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.299662 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:34:53.299758 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:34:53.299836 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":114,"threads_started":1}
I20260811 22:34:53.300473 20128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.292947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.18.211.130:33157)
0811 22:34:53.294580 (+ 1633us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.294596 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.294619 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:34:53.294695 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.294701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.294883 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.295045 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:53.295287 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.296461 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.298173 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.299081 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.299222 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.299351 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.300216 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.300329 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":120,"threads_started":1}
I20260811 22:34:53.302465 19521 tablet_service.cc:1467] Processing CreateTablet for tablet 5ccfec507b1448e2aafbab4fa187f595 (DEFAULT_TABLE table=test-table [id=bd4cb6851b1548559b95e7f2b634abf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:53.302598 19976 tablet_service.cc:1467] Processing CreateTablet for tablet 5ccfec507b1448e2aafbab4fa187f595 (DEFAULT_TABLE table=test-table [id=bd4cb6851b1548559b95e7f2b634abf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:53.303120 19741 tablet_service.cc:1467] Processing CreateTablet for tablet 5ccfec507b1448e2aafbab4fa187f595 (DEFAULT_TABLE table=test-table [id=bd4cb6851b1548559b95e7f2b634abf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:53.303618 19521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ccfec507b1448e2aafbab4fa187f595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:53.303711 19976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ccfec507b1448e2aafbab4fa187f595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:53.304090 19741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ccfec507b1448e2aafbab4fa187f595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:53.310088 20132 tablet_bootstrap.cc:492] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998: Bootstrap starting.
I20260811 22:34:53.311638 20138 tablet_bootstrap.cc:492] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Bootstrap starting.
I20260811 22:34:53.312443 20135 tablet_bootstrap.cc:492] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Bootstrap starting.
I20260811 22:34:53.312537 20132 tablet_bootstrap.cc:654] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:53.313463 20132 log.cc:824] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:53.313959 20138 tablet_bootstrap.cc:654] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:53.314561 20135 tablet_bootstrap.cc:654] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:53.315279 20138 log.cc:824] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:53.315725 20135 log.cc:824] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:53.315948 20132 tablet_bootstrap.cc:492] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998: No bootstrap required, opened a new log
I20260811 22:34:53.316167 20132 ts_tablet_manager.cc:1397] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:34:53.317430 20138 tablet_bootstrap.cc:492] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: No bootstrap required, opened a new log
I20260811 22:34:53.317581 20135 tablet_bootstrap.cc:492] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: No bootstrap required, opened a new log
I20260811 22:34:53.317623 20138 ts_tablet_manager.cc:1397] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:34:53.317790 20135 ts_tablet_manager.cc:1397] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:34:53.320776 20132 raft_consensus.cc:348] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.320777 20135 raft_consensus.cc:348] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.321080 20132 raft_consensus.cc:374] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:53.321082 20135 raft_consensus.cc:374] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:53.321133 20132 raft_consensus.cc:729] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9744862068344e8a6530b1753ede998, State: Initialized, Role: FOLLOWER
I20260811 22:34:53.321142 20135 raft_consensus.cc:729] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d245ada7bb8a40b1af8a75c81bf5a1d8, State: Initialized, Role: FOLLOWER
I20260811 22:34:53.321506 20138 raft_consensus.cc:348] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.321666 20132 consensus_queue.cc:260] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.321743 20138 raft_consensus.cc:374] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:53.321789 20138 raft_consensus.cc:729] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de6aa256d584aa48bd4fac41a182855, State: Initialized, Role: FOLLOWER
I20260811 22:34:53.321668 20135 consensus_queue.cc:260] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.322206 20138 consensus_queue.cc:260] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.322722 20132 ts_tablet_manager.cc:1428] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:34:53.322739 20135 ts_tablet_manager.cc:1428] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:34:53.322856 19858 heartbeater.cc:503] Master 127.18.211.190:35003 was elected leader, sending a full tablet report...
I20260811 22:34:53.323067 20138 ts_tablet_manager.cc:1428] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:34:53.323323 20099 heartbeater.cc:503] Master 127.18.211.190:35003 was elected leader, sending a full tablet report...
I20260811 22:34:53.323428 19634 heartbeater.cc:503] Master 127.18.211.190:35003 was elected leader, sending a full tablet report...
I20260811 22:34:53.395816 20147 raft_consensus.cc:482] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:53.396060 20147 raft_consensus.cc:504] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.397289 20147 leader_election.cc:288] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d245ada7bb8a40b1af8a75c81bf5a1d8 (127.18.211.130:33157), c9744862068344e8a6530b1753ede998 (127.18.211.129:32929)
I20260811 22:34:53.401194 20128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.397657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:56907 (local address 127.18.211.130:33157)
0811 22:34:53.397799 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.397804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.397821 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:34:53.397927 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.397932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.397986 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.398116 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:53.398438 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.398985 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.400109 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.400608 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.400744 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.400801 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.400992 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.401077 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:34:53.401194 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.397526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:33157 (local address 127.18.211.131:56907)
0811 22:34:53.397675 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.397706 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:53.397819 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:53.398123 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:53.398127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:53.398145 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:53.398320 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.398326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.399102 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.399109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.399973 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.399983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.400733 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.400738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.400853 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.400879 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:34:53.400972 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:34:53.401048 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:34:53.401593 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.398010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:45703 (local address 127.18.211.129:32929)
0811 22:34:53.398220 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.398225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.398350 (+ 125us) server_negotiation.cc:407] Connection header received
0811 22:34:53.398506 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.398510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.398563 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.398643 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:53.398909 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.399399 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.400455 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.400952 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.401101 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.401150 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.401392 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.401478 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 22:34:53.402012 20150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.397868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:32929 (local address 127.18.211.131:45703)
0811 22:34:53.398250 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.398265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:53.398394 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:53.398655 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:53.398657 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:53.398673 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:53.398802 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.398807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.399512 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.399515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.400336 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:53.400346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.401091 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.401095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:53.401202 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.401253 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:34:53.401815 (+ 562us) client_negotiation.cc:240] Negotiation successful
0811 22:34:53.401877 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260811 22:34:53.401937 19781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccfec507b1448e2aafbab4fa187f595" candidate_uuid: "5de6aa256d584aa48bd4fac41a182855" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" is_pre_election: true
I20260811 22:34:53.402279 19781 raft_consensus.cc:2454] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5de6aa256d584aa48bd4fac41a182855 in term 0.
I20260811 22:34:53.402841 19902 leader_election.cc:302] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de6aa256d584aa48bd4fac41a182855, d245ada7bb8a40b1af8a75c81bf5a1d8; no voters:
I20260811 22:34:53.403147 19551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccfec507b1448e2aafbab4fa187f595" candidate_uuid: "5de6aa256d584aa48bd4fac41a182855" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9744862068344e8a6530b1753ede998" is_pre_election: true
I20260811 22:34:53.403185 20147 raft_consensus.cc:2781] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:34:53.403275 20147 raft_consensus.cc:482] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:34:53.403347 20147 raft_consensus.cc:3037] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:53.403460 19551 raft_consensus.cc:2454] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5de6aa256d584aa48bd4fac41a182855 in term 0.
I20260811 22:34:53.404738 20147 raft_consensus.cc:504] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.405144 20147 leader_election.cc:288] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [CANDIDATE]: Term 1 election: Requested vote from peers d245ada7bb8a40b1af8a75c81bf5a1d8 (127.18.211.130:33157), c9744862068344e8a6530b1753ede998 (127.18.211.129:32929)
I20260811 22:34:53.405545 19781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccfec507b1448e2aafbab4fa187f595" candidate_uuid: "5de6aa256d584aa48bd4fac41a182855" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8"
I20260811 22:34:53.405565 19551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccfec507b1448e2aafbab4fa187f595" candidate_uuid: "5de6aa256d584aa48bd4fac41a182855" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9744862068344e8a6530b1753ede998"
I20260811 22:34:53.405660 19781 raft_consensus.cc:3037] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:53.405704 19551 raft_consensus.cc:3037] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:53.406752 19781 raft_consensus.cc:2454] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5de6aa256d584aa48bd4fac41a182855 in term 1.
I20260811 22:34:53.406826 19551 raft_consensus.cc:2454] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5de6aa256d584aa48bd4fac41a182855 in term 1.
I20260811 22:34:53.407053 19902 leader_election.cc:302] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de6aa256d584aa48bd4fac41a182855, d245ada7bb8a40b1af8a75c81bf5a1d8; no voters:
I20260811 22:34:53.407226 20147 raft_consensus.cc:2781] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:53.407469 20147 raft_consensus.cc:686] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 1 LEADER]: Becoming Leader. State: Replica: 5de6aa256d584aa48bd4fac41a182855, State: Running, Role: LEADER
I20260811 22:34:53.407819 20147 consensus_queue.cc:237] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } }
I20260811 22:34:53.411378 19296 catalog_manager.cc:5495] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5de6aa256d584aa48bd4fac41a182855 (127.18.211.131). New cstate: current_term: 1 leader_uuid: "5de6aa256d584aa48bd4fac41a182855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de6aa256d584aa48bd4fac41a182855" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 35571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 } health_report { overall_health: UNKNOWN } } }
I20260811 22:34:53.451279 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.447616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.18.211.131:35571)
0811 22:34:53.447762 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.447769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.447786 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:34:53.447876 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.447878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.447930 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.448053 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:53.448317 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.448829 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.449745 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.449963 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.450079 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.450400 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:53.450820 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:34:53.450990 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.451077 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.451120 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:34:53.457677 19551 raft_consensus.cc:1261] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 1 FOLLOWER]: Refusing update from remote peer 5de6aa256d584aa48bd4fac41a182855: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:53.457718 19781 raft_consensus.cc:1261] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 1 FOLLOWER]: Refusing update from remote peer 5de6aa256d584aa48bd4fac41a182855: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:53.458411 20147 consensus_queue.cc:1035] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9744862068344e8a6530b1753ede998" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:53.458601 20152 consensus_queue.cc:1035] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d245ada7bb8a40b1af8a75c81bf5a1d8" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 33157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:53.467157 20159 mvcc.cc:205] Tried to move back new op lower bound from 7317453592394903552 to 7317453592201379840. Current Snapshot: MvccSnapshot[applied={T|T < 7317453592394903552}]
I20260811 22:34:53.469622 20161 mvcc.cc:205] Tried to move back new op lower bound from 7317453592394903552 to 7317453592201379840. Current Snapshot: MvccSnapshot[applied={T|T < 7317453592394903552}]
I20260811 22:34:53.475620 19299 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:53.475804 19299 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:53.478545 19299 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c5a0dba9fe340fe9b2435b89f861521: Sending DeleteTablet for 3 replicas of tablet 5ccfec507b1448e2aafbab4fa187f595
I20260811 22:34:53.479393 19976 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ccfec507b1448e2aafbab4fa187f595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:53 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:52066
I20260811 22:34:53.480322 19741 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ccfec507b1448e2aafbab4fa187f595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:53 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:45768
I20260811 22:34:53.480429 19521 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ccfec507b1448e2aafbab4fa187f595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:53 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:55484
I20260811 22:34:53.480996 20178 tablet_replica.cc:330] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998: stopping tablet replica
I20260811 22:34:53.481367 20178 raft_consensus.cc:2229] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:53.481563 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19420
I20260811 22:34:53.481781 20178 raft_consensus.cc:2258] T 5ccfec507b1448e2aafbab4fa187f595 P c9744862068344e8a6530b1753ede998 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:53.482297 20174 tablet_replica.cc:330] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: stopping tablet replica
I20260811 22:34:53.485054 20174 raft_consensus.cc:2229] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:34:53.485458 20174 raft_consensus.cc:2258] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:53.487085 20174 ts_tablet_manager.cc:1905] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:53.490116 20174 ts_tablet_manager.cc:1918] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:53.490283 20174 log.cc:1196] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/5ccfec507b1448e2aafbab4fa187f595
I20260811 22:34:53.490712 20174 ts_tablet_manager.cc:1939] T 5ccfec507b1448e2aafbab4fa187f595 P 5de6aa256d584aa48bd4fac41a182855: Deleting consensus metadata
I20260811 22:34:53.492168 19263 catalog_manager.cc:4841] TS 5de6aa256d584aa48bd4fac41a182855 (127.18.211.131:35571): tablet 5ccfec507b1448e2aafbab4fa187f595 (table test-table [id=bd4cb6851b1548559b95e7f2b634abf0]) successfully deleted
I20260811 22:34:53.492779 20177 tablet_replica.cc:330] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: stopping tablet replica
I20260811 22:34:53.493139 20177 raft_consensus.cc:2229] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:53.493534 20177 raft_consensus.cc:2258] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:53.495229 20177 ts_tablet_manager.cc:1905] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:53.497463 20177 ts_tablet_manager.cc:1918] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:53.497579 20177 log.cc:1196] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/5ccfec507b1448e2aafbab4fa187f595
I20260811 22:34:53.497902 20177 ts_tablet_manager.cc:1939] T 5ccfec507b1448e2aafbab4fa187f595 P d245ada7bb8a40b1af8a75c81bf5a1d8: Deleting consensus metadata
I20260811 22:34:53.499037 19263 catalog_manager.cc:4841] TS d245ada7bb8a40b1af8a75c81bf5a1d8 (127.18.211.130:33157): tablet 5ccfec507b1448e2aafbab4fa187f595 (table test-table [id=bd4cb6851b1548559b95e7f2b634abf0]) successfully deleted
W20260811 22:34:53.500142 19263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:32929 (error 108)
I20260811 22:34:53.501075 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19641
I20260811 22:34:53.501276 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.500946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:32929 (local address 127.0.0.1:55486)
0811 22:34:53.501070 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:53.501157 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:32929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:34:53.503959 19263 catalog_manager.cc:4584] TS c9744862068344e8a6530b1753ede998 (127.18.211.129:32929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ccfec507b1448e2aafbab4fa187f595: Network error: Client connection negotiation failed: client connection to 127.18.211.129:32929: connect: Connection refused (error 111)
I20260811 22:34:53.524708 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19866
I20260811 22:34:53.540755 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 19227
2026-08-11T22:34:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2682 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T22:34:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:34:53Z Disabled control of system clock
I20260811 22:34:53.579490 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:38649
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:38649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:34:53.695729 20188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:53.695977 20188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:53.696019 20188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:53.699939 20188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:53.699988 20188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:53.700008 20188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:53.700026 20188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:53.700040 20188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:53.704879 20188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:38649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:38649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:53.705896 20188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:53.707239 20188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:53.715813 20206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:53.715816 20196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:53.715852 20188 server_base.cc:1034] running on GCE node
W20260811 22:34:53.715827 20198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:53.716667 20188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:53.717813 20188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:53.718988 20188 hybrid_clock.cc:648] HybridClock initialized: now 1786487693718967 us; error 54 us; skew 500 ppm
I20260811 22:34:53.725139 20188 webserver.cc:460] Webserver started at http://127.18.211.190:43763/ using document root <none> and password file <none>
I20260811 22:34:53.725780 20188 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:53.725875 20188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:53.726130 20188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:53.728197 20188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "c3fc6e334522471e82232c5fae5fd428"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "e85e9a60636a8d905140c955a9b4a7c3"
server_key_iv: "cabc775106449039a3641ba0e7b24241"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.728768 20188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "c3fc6e334522471e82232c5fae5fd428"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "e85e9a60636a8d905140c955a9b4a7c3"
server_key_iv: "cabc775106449039a3641ba0e7b24241"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.732550 20188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:34:53.734973 20214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:53.736202 20188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:53.736310 20188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "c3fc6e334522471e82232c5fae5fd428"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "e85e9a60636a8d905140c955a9b4a7c3"
server_key_iv: "cabc775106449039a3641ba0e7b24241"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.736393 20188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:53.756493 20188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:53.757087 20188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:53.757306 20188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:53.764422 20188 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:38649
I20260811 22:34:53.764456 20306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:38649 every 8 connection(s)
I20260811 22:34:53.765452 20188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:53.765764 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20188
I20260811 22:34:53.765892 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:34:53.766300 19278 external_mini_cluster.cc:1428] Setting key c274b04a4940a7ba7b6ae37f839e8de9
I20260811 22:34:53.769124 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:53.774875 20308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428: Bootstrap starting.
I20260811 22:34:53.776144 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:53.767759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58698 (local address 127.18.211.190:38649)
0811 22:34:53.768478 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:53.768495 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:53.768533 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:34:53.769250 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:53.769281 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:53.769718 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:53.770116 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:53.770540 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.771552 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.772631 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:53.773151 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:53.773376 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:53.774376 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:53.774381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:53.774395 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:53.774796 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:53.774853 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:53.774874 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:53.774989 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:53.775159 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:34:53.775421 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":140,"threads_started":1}
I20260811 22:34:53.777333 20308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:53.778239 20308 log.cc:824] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:53.780076 20308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428: No bootstrap required, opened a new log
I20260811 22:34:53.783522 20308 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc6e334522471e82232c5fae5fd428" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 38649 } }
I20260811 22:34:53.783778 20308 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:53.783833 20308 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3fc6e334522471e82232c5fae5fd428, State: Initialized, Role: FOLLOWER
I20260811 22:34:53.784315 20308 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc6e334522471e82232c5fae5fd428" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 38649 } }
I20260811 22:34:53.784503 20308 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:53.784578 20308 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:53.784663 20308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:53.785609 20308 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc6e334522471e82232c5fae5fd428" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 38649 } }
I20260811 22:34:53.785969 20308 leader_election.cc:302] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3fc6e334522471e82232c5fae5fd428; no voters:
I20260811 22:34:53.786331 20308 leader_election.cc:288] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:34:53.786456 20317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:53.786700 20317 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [term 1 LEADER]: Becoming Leader. State: Replica: c3fc6e334522471e82232c5fae5fd428, State: Running, Role: LEADER
I20260811 22:34:53.787007 20317 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc6e334522471e82232c5fae5fd428" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 38649 } }
I20260811 22:34:53.787551 20308 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:53.788928 20320 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3fc6e334522471e82232c5fae5fd428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc6e334522471e82232c5fae5fd428" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 38649 } } }
I20260811 22:34:53.788952 20321 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3fc6e334522471e82232c5fae5fd428. Latest consensus state: current_term: 1 leader_uuid: "c3fc6e334522471e82232c5fae5fd428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc6e334522471e82232c5fae5fd428" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 38649 } } }
I20260811 22:34:53.789067 20321 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:53.789065 20320 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428 [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:53.789572 20332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:53.792342 20332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:53.797536 20332 catalog_manager.cc:1347] Generated new cluster ID: 12e67054c5b74b2a9b8fa154f8a4b0e2
I20260811 22:34:53.797605 20332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:53.805364 20332 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:34:53.806583 20332 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:53.817126 20332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428: Generated new TSK 0
I20260811 22:34:53.817905 20332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:34:53.821455 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:38649
--builtin_ntp_servers=127.18.211.148:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:34:53.936131 20353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:53.936406 20353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:53.936445 20353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:53.940687 20353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:53.940762 20353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:53.940819 20353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:53.945815 20353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:38649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:53.946949 20353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:53.948390 20353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:53.958278 20361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:53.958272 20358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:53.958272 20359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:53.958654 20353 server_base.cc:1034] running on GCE node
I20260811 22:34:53.959306 20353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:53.959983 20353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:53.961146 20353 hybrid_clock.cc:648] HybridClock initialized: now 1786487693961112 us; error 46 us; skew 500 ppm
I20260811 22:34:53.963222 20353 webserver.cc:460] Webserver started at http://127.18.211.129:42005/ using document root <none> and password file <none>
I20260811 22:34:53.963907 20353 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:53.963974 20353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:53.964151 20353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:53.966357 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "92295972a219405f917fd8b4bf3827e0"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "96ebb58175cc299326250add78c20be8"
server_key_iv: "ca93227f1c3568587413c73cfc71d2ec"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.966929 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "92295972a219405f917fd8b4bf3827e0"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "96ebb58175cc299326250add78c20be8"
server_key_iv: "ca93227f1c3568587413c73cfc71d2ec"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.970860 20353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:34:53.973393 20370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:53.974687 20353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:53.974829 20353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "92295972a219405f917fd8b4bf3827e0"
format_stamp: "Formatted at 2026-08-11 22:34:53 on dist-test-slave-8hhm"
server_key: "96ebb58175cc299326250add78c20be8"
server_key_iv: "ca93227f1c3568587413c73cfc71d2ec"
server_key_version: "encryptionkey@0"
I20260811 22:34:53.974941 20353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:54.001034 20353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:54.001856 20353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:54.002079 20353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:54.002735 20353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:54.003841 20353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:54.003892 20353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.003958 20353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:54.003988 20353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.013732 20353 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:43505
I20260811 22:34:54.013760 20571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:43505 every 8 connection(s)
I20260811 22:34:54.014725 20353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:54.019450 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20353
I20260811 22:34:54.019619 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:34:54.019989 19278 external_mini_cluster.cc:1428] Setting key bcc19fab5fe603b90c0f20f752e821c2
I20260811 22:34:54.022359 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:38649
--builtin_ntp_servers=127.18.211.148:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:34:54.026176 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.016764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:53385 (local address 127.18.211.190:38649)
0811 22:34:54.016911 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.016915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.018114 (+ 1199us) server_negotiation.cc:407] Connection header received
0811 22:34:54.020075 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.020083 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.020154 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.020233 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:54.021358 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.022019 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.024025 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.024258 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.024378 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.025622 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:54.025626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:54.025629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:54.025647 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:54.025667 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:54.025674 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:54.025748 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.025924 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.026027 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:34:54.027047 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.017190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:38649 (local address 127.18.211.129:53385)
0811 22:34:54.017937 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.017983 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.019788 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.020485 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.020501 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.020607 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:54.021157 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.021173 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.022174 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.022181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.023879 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.023896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.024426 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.024431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.024631 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.025332 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:54.025334 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:54.025378 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:54.025446 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:54.025764 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:54.025774 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:54.025780 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.025908 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.026212 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":110,"threads_started":1}
I20260811 22:34:54.028673 20572 heartbeater.cc:347] Connected to a master server at 127.18.211.190:38649
I20260811 22:34:54.029048 20572 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:54.029693 20572 heartbeater.cc:511] Master 127.18.211.190:38649 requested a full tablet report, sending...
I20260811 22:34:54.031433 20244 ts_manager.cc:194] Registered new tserver with Master: 92295972a219405f917fd8b4bf3827e0 (127.18.211.129:43505)
I20260811 22:34:54.032677 20244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.129:53385
W20260811 22:34:54.150823 20576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:54.151132 20576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:54.151173 20576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:54.155717 20576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:54.155791 20576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:54.155851 20576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:54.160987 20576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:38649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:54.162262 20576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:54.163713 20576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:54.174098 20587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:54.174098 20584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:54.174167 20585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:54.174592 20576 server_base.cc:1034] running on GCE node
I20260811 22:34:54.175047 20576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:54.175782 20576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:54.176993 20576 hybrid_clock.cc:648] HybridClock initialized: now 1786487694176969 us; error 36 us; skew 500 ppm
I20260811 22:34:54.179363 20576 webserver.cc:460] Webserver started at http://127.18.211.130:35441/ using document root <none> and password file <none>
I20260811 22:34:54.180015 20576 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:54.180119 20576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:54.180367 20576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:54.182727 20576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "62da6bd7738941ec9f12bffffcc8cab1"
format_stamp: "Formatted at 2026-08-11 22:34:54 on dist-test-slave-8hhm"
server_key: "328344ea6d8b323783ee96195d21a177"
server_key_iv: "330789e7096efd7c9aa46a818a0adb55"
server_key_version: "encryptionkey@0"
I20260811 22:34:54.183336 20576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "62da6bd7738941ec9f12bffffcc8cab1"
format_stamp: "Formatted at 2026-08-11 22:34:54 on dist-test-slave-8hhm"
server_key: "328344ea6d8b323783ee96195d21a177"
server_key_iv: "330789e7096efd7c9aa46a818a0adb55"
server_key_version: "encryptionkey@0"
I20260811 22:34:54.187479 20576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:34:54.190838 20597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.192294 20576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:34:54.192553 20576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "62da6bd7738941ec9f12bffffcc8cab1"
format_stamp: "Formatted at 2026-08-11 22:34:54 on dist-test-slave-8hhm"
server_key: "328344ea6d8b323783ee96195d21a177"
server_key_iv: "330789e7096efd7c9aa46a818a0adb55"
server_key_version: "encryptionkey@0"
I20260811 22:34:54.192692 20576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:54.219630 20576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:54.220471 20576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:54.220701 20576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:54.221443 20576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:54.222522 20576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:54.222613 20576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.222697 20576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:54.222740 20576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.232020 20576 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:39247
I20260811 22:34:54.232043 20806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:39247 every 8 connection(s)
I20260811 22:34:54.233039 20576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:34:54.241972 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20576
I20260811 22:34:54.242120 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:34:54.242503 19278 external_mini_cluster.cc:1428] Setting key 18a96ec047a1181da9c4bc33770b8b5d
I20260811 22:34:54.243322 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.235062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:46275 (local address 127.18.211.190:38649)
0811 22:34:54.235215 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.235219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.236427 (+ 1208us) server_negotiation.cc:407] Connection header received
0811 22:34:54.237939 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.237941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.237995 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.238120 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:54.239104 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.239628 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.240966 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.241320 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.241415 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.242799 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:54.242802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:54.242804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:54.242817 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:54.242837 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:54.242845 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:54.242907 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.243116 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.243195 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:34:54.244421 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.235560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:38649 (local address 127.18.211.130:46275)
0811 22:34:54.236261 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.236305 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.237716 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.238298 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.238305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.238395 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:54.238946 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.238959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.239743 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.239746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.240828 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.240839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.241468 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.241471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.241647 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.242465 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:54.242467 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:54.242527 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:54.242622 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:54.242938 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:54.242948 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:54.242956 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.243083 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.243391 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":127,"threads_started":1}
I20260811 22:34:54.244824 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:38649
--builtin_ntp_servers=127.18.211.148:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:34:54.245745 20807 heartbeater.cc:347] Connected to a master server at 127.18.211.190:38649
I20260811 22:34:54.246016 20807 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:54.246615 20807 heartbeater.cc:511] Master 127.18.211.190:38649 requested a full tablet report, sending...
I20260811 22:34:54.247776 20244 ts_manager.cc:194] Registered new tserver with Master: 62da6bd7738941ec9f12bffffcc8cab1 (127.18.211.130:39247)
I20260811 22:34:54.248585 20244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.130:46275
W20260811 22:34:54.367929 20813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:54.368207 20813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:54.368247 20813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:54.372248 20813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:54.372309 20813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:54.372364 20813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:54.377064 20813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:38649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:54.378235 20813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:54.379717 20813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:54.389403 20820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:54.389429 20825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:54.389410 20819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:54.389602 20813 server_base.cc:1034] running on GCE node
I20260811 22:34:54.390152 20813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:54.390828 20813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:54.391999 20813 hybrid_clock.cc:648] HybridClock initialized: now 1786487694391978 us; error 32 us; skew 500 ppm
I20260811 22:34:54.393940 20813 webserver.cc:460] Webserver started at http://127.18.211.131:41917/ using document root <none> and password file <none>
I20260811 22:34:54.394546 20813 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:54.394627 20813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:54.394873 20813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:54.397004 20813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "3eaafb50ff1f481e81a824a0d3359e47"
format_stamp: "Formatted at 2026-08-11 22:34:54 on dist-test-slave-8hhm"
server_key: "c00f7c1625a7319ee619831f0473b2bf"
server_key_iv: "f90b89948f95333228551aef6b24a68b"
server_key_version: "encryptionkey@0"
I20260811 22:34:54.397636 20813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "3eaafb50ff1f481e81a824a0d3359e47"
format_stamp: "Formatted at 2026-08-11 22:34:54 on dist-test-slave-8hhm"
server_key: "c00f7c1625a7319ee619831f0473b2bf"
server_key_iv: "f90b89948f95333228551aef6b24a68b"
server_key_version: "encryptionkey@0"
I20260811 22:34:54.401495 20813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:34:54.403977 20838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.405253 20813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:34:54.405397 20813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "3eaafb50ff1f481e81a824a0d3359e47"
format_stamp: "Formatted at 2026-08-11 22:34:54 on dist-test-slave-8hhm"
server_key: "c00f7c1625a7319ee619831f0473b2bf"
server_key_iv: "f90b89948f95333228551aef6b24a68b"
server_key_version: "encryptionkey@0"
I20260811 22:34:54.405501 20813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:54.427560 20813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:54.428303 20813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:54.428504 20813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:54.429133 20813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:54.430107 20813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:54.430153 20813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.430217 20813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:54.430250 20813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:54.439284 20813 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:41931
I20260811 22:34:54.439304 21047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:41931 every 8 connection(s)
I20260811 22:34:54.440343 20813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:54.443987 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 20813
I20260811 22:34:54.444114 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:34:54.444432 19278 external_mini_cluster.cc:1428] Setting key ea25563c0f8d1bb4cc33a9352e599895
I20260811 22:34:54.450989 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.442414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:35813 (local address 127.18.211.190:38649)
0811 22:34:54.442629 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.442636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.443905 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 22:34:54.445875 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.445879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.445967 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.446071 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:54.447180 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.447773 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.448929 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.449184 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.449364 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.450503 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:54.450507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:54.450509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:54.450524 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:54.450550 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:54.450559 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:54.450637 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.450812 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.450869 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 22:34:54.451891 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.442948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:38649 (local address 127.18.211.131:35813)
0811 22:34:54.443705 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.443767 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.445614 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.446276 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.446284 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.446373 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:54.447033 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.447049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.447894 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.447896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.448788 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.448797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.449406 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.449409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.449561 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.450180 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:54.450181 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:54.450231 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:54.450300 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:54.450647 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:54.450654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:54.450659 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.450761 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.450999 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":154,"threads_started":1}
I20260811 22:34:54.453117 21048 heartbeater.cc:347] Connected to a master server at 127.18.211.190:38649
I20260811 22:34:54.453397 21048 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:54.453949 21048 heartbeater.cc:511] Master 127.18.211.190:38649 requested a full tablet report, sending...
I20260811 22:34:54.455102 20244 ts_manager.cc:194] Registered new tserver with Master: 3eaafb50ff1f481e81a824a0d3359e47 (127.18.211.131:41931)
I20260811 22:34:54.455693 20244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.131:35813
I20260811 22:34:54.458766 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:34:54.464871 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.460789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.18.211.190:38649)
0811 22:34:54.460951 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.460958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.461173 (+ 215us) server_negotiation.cc:407] Connection header received
0811 22:34:54.461373 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.461375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.461417 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.461498 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:54.461798 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.462301 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.463153 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.463375 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.463484 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.464351 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:54.464359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:54.464362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:54.464373 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:54.464392 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:54.464400 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:54.464528 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.464676 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.464726 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:34:54.467713 20244 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:34:54.470811 20244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:34:54.486105 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.480561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48056 (local address 127.18.211.130:39247)
0811 22:34:54.480877 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.480890 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.481092 (+ 202us) server_negotiation.cc:407] Connection header received
0811 22:34:54.481382 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.481388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.481588 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.481733 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:54.482144 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.482878 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.484555 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.485428 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.485553 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.485770 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.485823 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.485924 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":94,"threads_started":1}
I20260811 22:34:54.486100 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.480466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:41931 (local address 127.0.0.1:58916)
0811 22:34:54.481026 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.481051 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.481143 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.481832 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.481838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.481860 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:54.481952 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.481957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.483445 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.483450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.484593 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.484605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.485673 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.485676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.485778 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.485807 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.485858 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.485924 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
I20260811 22:34:54.486121 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.479936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:39247 (local address 127.0.0.1:48056)
0811 22:34:54.480957 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.480978 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.481166 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.481913 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.481917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.481935 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:54.482030 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.482036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.482996 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.483006 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.484401 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.484413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.485555 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.485558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.485645 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.485662 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.485718 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.485987 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"mutex_wait_us":50,"thread_start_us":88,"threads_started":1}
I20260811 22:34:54.486433 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.479757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:43505 (local address 127.0.0.1:44642)
0811 22:34:54.480261 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.480309 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.480446 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.481730 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.481736 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.481760 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:54.481904 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.481912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.483341 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.483344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.484500 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.484510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.485430 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.485434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.485556 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.485609 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.485703 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.485870 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":29,"thread_start_us":92,"threads_started":1}
I20260811 22:34:54.486677 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.479880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.18.211.129:43505)
0811 22:34:54.481171 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.481180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.481206 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:34:54.481315 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.481320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.481548 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.481709 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:54.482026 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.483224 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.484622 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.485286 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.485878 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.486060 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.486111 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.486414 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1187,"thread_start_us":119,"threads_started":1}
I20260811 22:34:54.487082 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.480646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.18.211.131:41931)
0811 22:34:54.481417 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.481428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.481448 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:34:54.481514 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.481519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.481685 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.482138 (+ 453us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:54.482276 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.483335 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.484732 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.485537 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.486621 (+ 1084us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.486791 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.486838 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.486927 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":163,"threads_started":1}
I20260811 22:34:54.489576 20699 tablet_service.cc:1467] Processing CreateTablet for tablet e593d936f4c341a085ee9c1fd553d6bc (DEFAULT_TABLE table=test-table [id=a52db639e3364ee1b636750eef216a49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:54.489996 20930 tablet_service.cc:1467] Processing CreateTablet for tablet e593d936f4c341a085ee9c1fd553d6bc (DEFAULT_TABLE table=test-table [id=a52db639e3364ee1b636750eef216a49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:54.490783 20699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e593d936f4c341a085ee9c1fd553d6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:54.491060 20930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e593d936f4c341a085ee9c1fd553d6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:54.490851 20449 tablet_service.cc:1467] Processing CreateTablet for tablet e593d936f4c341a085ee9c1fd553d6bc (DEFAULT_TABLE table=test-table [id=a52db639e3364ee1b636750eef216a49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:54.491827 20449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e593d936f4c341a085ee9c1fd553d6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:54.498361 21083 tablet_bootstrap.cc:492] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Bootstrap starting.
I20260811 22:34:54.499166 21085 tablet_bootstrap.cc:492] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Bootstrap starting.
I20260811 22:34:54.500697 21083 tablet_bootstrap.cc:654] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:54.500869 21085 tablet_bootstrap.cc:654] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:54.501556 21083 log.cc:824] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:54.501564 21086 tablet_bootstrap.cc:492] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: Bootstrap starting.
I20260811 22:34:54.501705 21085 log.cc:824] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:54.503635 21086 tablet_bootstrap.cc:654] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:54.503800 21083 tablet_bootstrap.cc:492] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: No bootstrap required, opened a new log
I20260811 22:34:54.504004 21083 ts_tablet_manager.cc:1397] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260811 22:34:54.504483 21086 log.cc:824] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:54.505136 21085 tablet_bootstrap.cc:492] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: No bootstrap required, opened a new log
I20260811 22:34:54.505388 21085 ts_tablet_manager.cc:1397] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:34:54.506613 21086 tablet_bootstrap.cc:492] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: No bootstrap required, opened a new log
I20260811 22:34:54.506793 21086 ts_tablet_manager.cc:1397] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:34:54.508618 21085 raft_consensus.cc:348] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.508559 21083 raft_consensus.cc:348] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.508857 21083 raft_consensus.cc:374] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:54.508857 21085 raft_consensus.cc:374] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:54.508919 21085 raft_consensus.cc:729] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62da6bd7738941ec9f12bffffcc8cab1, State: Initialized, Role: FOLLOWER
I20260811 22:34:54.508919 21083 raft_consensus.cc:729] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eaafb50ff1f481e81a824a0d3359e47, State: Initialized, Role: FOLLOWER
I20260811 22:34:54.509454 21083 consensus_queue.cc:260] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.509459 21085 consensus_queue.cc:260] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.510483 21085 ts_tablet_manager.cc:1428] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:34:54.510509 21083 ts_tablet_manager.cc:1428] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:34:54.510880 20807 heartbeater.cc:503] Master 127.18.211.190:38649 was elected leader, sending a full tablet report...
I20260811 22:34:54.511081 21048 heartbeater.cc:503] Master 127.18.211.190:38649 was elected leader, sending a full tablet report...
I20260811 22:34:54.511013 21086 raft_consensus.cc:348] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.511269 21086 raft_consensus.cc:374] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:54.511323 21086 raft_consensus.cc:729] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92295972a219405f917fd8b4bf3827e0, State: Initialized, Role: FOLLOWER
I20260811 22:34:54.511801 21086 consensus_queue.cc:260] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.512642 21086 ts_tablet_manager.cc:1428] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:34:54.513257 20572 heartbeater.cc:503] Master 127.18.211.190:38649 was elected leader, sending a full tablet report...
W20260811 22:34:54.515884 20573 tablet.cc:2369] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:34:54.582753 21091 raft_consensus.cc:482] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:54.582983 21091 raft_consensus.cc:504] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.584216 21091 leader_election.cc:288] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92295972a219405f917fd8b4bf3827e0 (127.18.211.129:43505), 62da6bd7738941ec9f12bffffcc8cab1 (127.18.211.130:39247)
I20260811 22:34:54.588160 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.584423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:43505 (local address 127.18.211.131:41919)
0811 22:34:54.584630 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.584646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.584760 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.585066 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.585069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.585087 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:54.585263 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.585273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.586049 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.586052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.586951 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.586961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.587729 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.587733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.587845 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.587872 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.587953 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.588023 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":7}
I20260811 22:34:54.588193 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.584551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:41919 (local address 127.18.211.129:43505)
0811 22:34:54.584682 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.584687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.584706 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:34:54.584890 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.584897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.584961 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.585039 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:54.585379 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.585962 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.587075 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.587570 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.587728 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.587782 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.587994 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.588064 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:34:54.588455 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.584444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:39247 (local address 127.18.211.131:51437)
0811 22:34:54.584854 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.584867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:54.584966 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:54.585284 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:54.585286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:54.585300 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:54.585430 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.585436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.586356 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.586359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.587149 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:54.587159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.588018 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.588021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:54.588123 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.588145 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:34:54.588222 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:34:54.588287 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":142,"threads_started":1}
I20260811 22:34:54.588771 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.584569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:51437 (local address 127.18.211.130:39247)
0811 22:34:54.584697 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.584702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.584949 (+ 247us) server_negotiation.cc:407] Connection header received
0811 22:34:54.585069 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.585072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.585122 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.585263 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:54.585556 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.586243 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.587323 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.587877 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.587991 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.588032 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.588594 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.588671 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:34:54.588932 20495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593d936f4c341a085ee9c1fd553d6bc" candidate_uuid: "3eaafb50ff1f481e81a824a0d3359e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92295972a219405f917fd8b4bf3827e0" is_pre_election: true
I20260811 22:34:54.589303 20495 raft_consensus.cc:2454] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eaafb50ff1f481e81a824a0d3359e47 in term 0.
I20260811 22:34:54.589377 20729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593d936f4c341a085ee9c1fd553d6bc" candidate_uuid: "3eaafb50ff1f481e81a824a0d3359e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62da6bd7738941ec9f12bffffcc8cab1" is_pre_election: true
I20260811 22:34:54.589653 20729 raft_consensus.cc:2454] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eaafb50ff1f481e81a824a0d3359e47 in term 0.
I20260811 22:34:54.589833 20839 leader_election.cc:302] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eaafb50ff1f481e81a824a0d3359e47, 92295972a219405f917fd8b4bf3827e0; no voters:
I20260811 22:34:54.590131 21091 raft_consensus.cc:2781] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:34:54.590210 21091 raft_consensus.cc:482] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:34:54.590256 21091 raft_consensus.cc:3037] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:54.591292 21091 raft_consensus.cc:504] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.591674 21091 leader_election.cc:288] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [CANDIDATE]: Term 1 election: Requested vote from peers 92295972a219405f917fd8b4bf3827e0 (127.18.211.129:43505), 62da6bd7738941ec9f12bffffcc8cab1 (127.18.211.130:39247)
I20260811 22:34:54.592037 20495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593d936f4c341a085ee9c1fd553d6bc" candidate_uuid: "3eaafb50ff1f481e81a824a0d3359e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92295972a219405f917fd8b4bf3827e0"
I20260811 22:34:54.592051 20729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593d936f4c341a085ee9c1fd553d6bc" candidate_uuid: "3eaafb50ff1f481e81a824a0d3359e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62da6bd7738941ec9f12bffffcc8cab1"
I20260811 22:34:54.592166 20495 raft_consensus.cc:3037] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:54.592166 20729 raft_consensus.cc:3037] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:54.593466 20495 raft_consensus.cc:2454] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eaafb50ff1f481e81a824a0d3359e47 in term 1.
I20260811 22:34:54.593466 20729 raft_consensus.cc:2454] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eaafb50ff1f481e81a824a0d3359e47 in term 1.
I20260811 22:34:54.593791 20848 leader_election.cc:302] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eaafb50ff1f481e81a824a0d3359e47, 62da6bd7738941ec9f12bffffcc8cab1; no voters:
I20260811 22:34:54.593950 21091 raft_consensus.cc:2781] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:54.594179 21091 raft_consensus.cc:686] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 1 LEADER]: Becoming Leader. State: Replica: 3eaafb50ff1f481e81a824a0d3359e47, State: Running, Role: LEADER
I20260811 22:34:54.594547 21091 consensus_queue.cc:237] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } }
I20260811 22:34:54.597887 20244 catalog_manager.cc:5495] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eaafb50ff1f481e81a824a0d3359e47 (127.18.211.131). New cstate: current_term: 1 leader_uuid: "3eaafb50ff1f481e81a824a0d3359e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3eaafb50ff1f481e81a824a0d3359e47" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 41931 } health_report { overall_health: HEALTHY } } }
I20260811 22:34:54.612053 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.608366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58926 (local address 127.18.211.131:41931)
0811 22:34:54.608488 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:54.608492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:54.608506 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:34:54.608665 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:54.608668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:54.608728 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:54.608818 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:54.609040 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.609679 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.610595 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:54.610803 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:54.610878 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:54.611133 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:54.611549 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:34:54.611654 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:54.611820 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:34:54.611874 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:34:54.618137 20729 raft_consensus.cc:1261] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 1 FOLLOWER]: Refusing update from remote peer 3eaafb50ff1f481e81a824a0d3359e47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:54.618137 20495 raft_consensus.cc:1261] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 1 FOLLOWER]: Refusing update from remote peer 3eaafb50ff1f481e81a824a0d3359e47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:54.618835 21091 consensus_queue.cc:1035] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62da6bd7738941ec9f12bffffcc8cab1" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 39247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:54.619081 21104 consensus_queue.cc:1035] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92295972a219405f917fd8b4bf3827e0" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 43505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:54.628991 21111 mvcc.cc:205] Tried to move back new op lower bound from 7317453597148446720 to 7317453597062049792. Current Snapshot: MvccSnapshot[applied={T|T < 7317453597148446720}]
I20260811 22:34:54.635149 20244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:54.635345 20244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:54.637861 20244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3fc6e334522471e82232c5fae5fd428: Sending DeleteTablet for 3 replicas of tablet e593d936f4c341a085ee9c1fd553d6bc
I20260811 22:34:54.638733 20699 tablet_service.cc:1514] Processing DeleteTablet for tablet e593d936f4c341a085ee9c1fd553d6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:54 UTC) from {username='slave'} at 127.0.0.1:48056
I20260811 22:34:54.638743 20930 tablet_service.cc:1514] Processing DeleteTablet for tablet e593d936f4c341a085ee9c1fd553d6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:54 UTC) from {username='slave'} at 127.0.0.1:58916
I20260811 22:34:54.638777 20449 tablet_service.cc:1514] Processing DeleteTablet for tablet e593d936f4c341a085ee9c1fd553d6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:54 UTC) from {username='slave'} at 127.0.0.1:44642
I20260811 22:34:54.639333 21129 tablet_replica.cc:330] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0: stopping tablet replica
I20260811 22:34:54.639647 21130 tablet_replica.cc:330] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: stopping tablet replica
I20260811 22:34:54.639793 21128 tablet_replica.cc:330] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: stopping tablet replica
I20260811 22:34:54.639955 21130 raft_consensus.cc:2229] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:34:54.640066 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20353
I20260811 22:34:54.640175 21128 raft_consensus.cc:2229] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:54.640329 21129 raft_consensus.cc:2229] T e593d936f4c341a085ee9c1fd553d6bc P 92295972a219405f917fd8b4bf3827e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:54.640522 21128 raft_consensus.cc:2258] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:54.640882 21130 raft_consensus.cc:2258] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:54.656255 21128 ts_tablet_manager.cc:1905] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:54.659650 21128 ts_tablet_manager.cc:1918] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:54.659761 21128 log.cc:1196] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/e593d936f4c341a085ee9c1fd553d6bc
I20260811 22:34:54.660051 21128 ts_tablet_manager.cc:1939] T e593d936f4c341a085ee9c1fd553d6bc P 62da6bd7738941ec9f12bffffcc8cab1: Deleting consensus metadata
I20260811 22:34:54.661865 20221 catalog_manager.cc:4841] TS 62da6bd7738941ec9f12bffffcc8cab1 (127.18.211.130:39247): tablet e593d936f4c341a085ee9c1fd553d6bc (table test-table [id=a52db639e3364ee1b636750eef216a49]) successfully deleted
I20260811 22:34:54.679693 21130 ts_tablet_manager.cc:1905] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:54.682864 21130 ts_tablet_manager.cc:1918] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:54.682996 21130 log.cc:1196] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/e593d936f4c341a085ee9c1fd553d6bc
I20260811 22:34:54.683351 21130 ts_tablet_manager.cc:1939] T e593d936f4c341a085ee9c1fd553d6bc P 3eaafb50ff1f481e81a824a0d3359e47: Deleting consensus metadata
I20260811 22:34:54.684769 20215 catalog_manager.cc:4841] TS 3eaafb50ff1f481e81a824a0d3359e47 (127.18.211.131:41931): tablet e593d936f4c341a085ee9c1fd553d6bc (table test-table [id=a52db639e3364ee1b636750eef216a49]) successfully deleted
W20260811 22:34:54.685184 20215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:43505 (error 108)
I20260811 22:34:54.685473 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20576
I20260811 22:34:54.688783 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:54.688368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:43505 (local address 127.0.0.1:44644)
0811 22:34:54.688563 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:54.688659 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:43505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:34:54.688968 20215 catalog_manager.cc:4584] TS 92295972a219405f917fd8b4bf3827e0 (127.18.211.129:43505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e593d936f4c341a085ee9c1fd553d6bc: Network error: Client connection negotiation failed: client connection to 127.18.211.129:43505: connect: Connection refused (error 111)
I20260811 22:34:54.695703 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20813
I20260811 22:34:54.713920 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 20188
2026-08-11T22:34:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1179 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:34:54 dist-test-slave-8hhm krb5kdc[21142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:34:54 dist-test-slave-8hhm krb5kdc[21142](info): set up 2 sockets
Aug 11 22:34:54 dist-test-slave-8hhm krb5kdc[21142](info): commencing operation
krb5kdc: starting...
W20260811 22:34:56.031178 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:34:56 dist-test-slave-8hhm krb5kdc[21142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:34:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:34:56.168216 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43547
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:43547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:56.288326 21168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:56.288599 21168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:56.288640 21168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:56.292543 21168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:56.292601 21168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:56.292621 21168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:56.292637 21168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:56.292654 21168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:56.297816 21168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:43547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:43547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:56.298846 21168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:56.299824 21168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:56.308776 21179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:56.308734 21180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:56.308830 21168 server_base.cc:1034] running on GCE node
W20260811 22:34:56.308734 21182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:56.309626 21168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:56.310724 21168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:56.311884 21168 hybrid_clock.cc:648] HybridClock initialized: now 1786487696311856 us; error 41 us; skew 500 ppm
Aug 11 22:34:56 dist-test-slave-8hhm krb5kdc[21142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487696, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:56.314864 21168 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:34:56.315886 21168 webserver.cc:460] Webserver started at http://127.18.211.190:38675/ using document root <none> and password file <none>
I20260811 22:34:56.316483 21168 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:56.316548 21168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:56.316779 21168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:56.318969 21168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "79334718c0d5406886d41da508c960af"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "1e75b34c0aeaf534c9fa97e1da039d30"
server_key_iv: "be7c6a065e5863ad34c6800edcf3ae27"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.319576 21168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "79334718c0d5406886d41da508c960af"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "1e75b34c0aeaf534c9fa97e1da039d30"
server_key_iv: "be7c6a065e5863ad34c6800edcf3ae27"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.323513 21168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:34:56.326288 21194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:56.327566 21168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:34:56.327723 21168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "79334718c0d5406886d41da508c960af"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "1e75b34c0aeaf534c9fa97e1da039d30"
server_key_iv: "be7c6a065e5863ad34c6800edcf3ae27"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.327845 21168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:56.338281 21168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:56.341435 21168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:56.341642 21168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:56.349042 21168 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:43547
I20260811 22:34:56.349054 21286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:43547 every 8 connection(s)
I20260811 22:34:56.350051 21168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:56.352880 21287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:56.355461 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21168
I20260811 22:34:56.355628 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:34:56.355932 19278 external_mini_cluster.cc:1428] Setting key 345f996620c0df1ee3d0bdcbf029b71a
I20260811 22:34:56.359266 21287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af: Bootstrap starting.
I20260811 22:34:56.362023 21287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:56.362843 21287 log.cc:824] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:56.364910 21287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af: No bootstrap required, opened a new log
Aug 11 22:34:56 dist-test-slave-8hhm krb5kdc[21142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:56.369068 21287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79334718c0d5406886d41da508c960af" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43547 } }
I20260811 22:34:56.369410 21287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:56.369473 21287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79334718c0d5406886d41da508c960af, State: Initialized, Role: FOLLOWER
I20260811 22:34:56.369979 21287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79334718c0d5406886d41da508c960af" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43547 } }
I20260811 22:34:56.370173 21287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:56.370249 21287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:56.370349 21287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:56.371464 21287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79334718c0d5406886d41da508c960af" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43547 } }
I20260811 22:34:56.371848 21287 leader_election.cc:302] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79334718c0d5406886d41da508c960af; no voters:
I20260811 22:34:56.372187 21287 leader_election.cc:288] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:34:56.372308 21294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:56.372414 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:56.357397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53936 (local address 127.18.211.190:43547)
0811 22:34:56.357808 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:56.357827 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:56.357873 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:34:56.358701 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:56.358725 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:56.359145 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:56.359512 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:56.360424 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.361934 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.362834 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.363469 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.363631 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:56.366192 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:56.366198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:56.366215 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:56.366246 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:56.369043 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:56.369595 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:56.369601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:56.369607 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:56.369746 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:56.370081 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:56.370084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:56.370086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:56.370631 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:56.370826 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:56.371079 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:34:56.371407 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":98,"threads_started":1}
I20260811 22:34:56.372591 21294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [term 1 LEADER]: Becoming Leader. State: Replica: 79334718c0d5406886d41da508c960af, State: Running, Role: LEADER
I20260811 22:34:56.372903 21294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79334718c0d5406886d41da508c960af" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43547 } }
I20260811 22:34:56.373473 21287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:56.375432 21295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79334718c0d5406886d41da508c960af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79334718c0d5406886d41da508c960af" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43547 } } }
I20260811 22:34:56.375566 21295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:56.375587 21296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79334718c0d5406886d41da508c960af. Latest consensus state: current_term: 1 leader_uuid: "79334718c0d5406886d41da508c960af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79334718c0d5406886d41da508c960af" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 43547 } } }
I20260811 22:34:56.375687 21296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af [sys.catalog]: This master's current role is: LEADER
W20260811 22:34:56.378825 21326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:34:56.378912 21326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:34:56.378983 21320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:56.379688 21320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:56.384888 21320 catalog_manager.cc:1347] Generated new cluster ID: 657cddb700874aebb08ceef45f1cc586
I20260811 22:34:56.384955 21320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:56.397667 21320 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:34:56.398602 21320 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:56.409613 21320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af: Generated new TSK 0
I20260811 22:34:56.410305 21320 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.211.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
Entry for principal kudu/127.18.211.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab.
I20260811 22:34:56.442720 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43547
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:56.563279 21334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:56.563555 21334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:56.563629 21334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:56.567677 21334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:56.567749 21334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:56.567840 21334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:56.573117 21334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.129.keytab
--principal=kudu/127.18.211.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:56.574352 21334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:56.575517 21334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:56.585423 21346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:56.585423 21341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:56.585507 21334 server_base.cc:1034] running on GCE node
W20260811 22:34:56.585427 21340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:56.586113 21334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:56.586778 21334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:56.588021 21334 hybrid_clock.cc:648] HybridClock initialized: now 1786487696588006 us; error 27 us; skew 500 ppm
Aug 11 22:34:56 dist-test-slave-8hhm krb5kdc[21142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487696, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:56.590957 21334 init.cc:377] Logged in from keytab as kudu/127.18.211.129@KRBTEST.COM (short username kudu)
I20260811 22:34:56.591941 21334 webserver.cc:460] Webserver started at http://127.18.211.129:38305/ using document root <none> and password file <none>
I20260811 22:34:56.592607 21334 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:56.592710 21334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:56.593037 21334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:56.595413 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "35862fdc92dd4f589b44e54dc18054ee"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "ab8dd2abf689eaa4fefc11f4da62363a"
server_key_iv: "68c2c3fa5d0cc8736e9baab4a5aaf7c3"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.595999 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "35862fdc92dd4f589b44e54dc18054ee"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "ab8dd2abf689eaa4fefc11f4da62363a"
server_key_iv: "68c2c3fa5d0cc8736e9baab4a5aaf7c3"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.600028 21334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:34:56.602655 21358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:56.603904 21334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:34:56.604048 21334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "35862fdc92dd4f589b44e54dc18054ee"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "ab8dd2abf689eaa4fefc11f4da62363a"
server_key_iv: "68c2c3fa5d0cc8736e9baab4a5aaf7c3"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.604146 21334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:56.633177 21334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:56.636195 21334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:56.636355 21334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:56.636940 21334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:56.637801 21334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:56.637850 21334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:56.637890 21334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:56.637912 21334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:56.646872 21334 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:41535
I20260811 22:34:56.646899 21569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:41535 every 8 connection(s)
I20260811 22:34:56.647787 21334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:56.652467 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21334
I20260811 22:34:56.652613 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:34:56.652944 19278 external_mini_cluster.cc:1428] Setting key 81a7f881dca3c08ed4d63bdef0481c10
Aug 11 22:34:56 dist-test-slave-8hhm krb5kdc[21142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487696, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.129@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:56.663798 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:56.649738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:60971 (local address 127.18.211.190:43547)
0811 22:34:56.649912 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:56.649917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:56.650791 (+ 874us) server_negotiation.cc:407] Connection header received
0811 22:34:56.652163 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:56.652167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:56.652219 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:56.652356 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:56.654003 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.654687 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.655895 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.656141 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.656270 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:56.659659 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:56.659662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:56.659664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:56.659688 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:56.661586 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:56.662162 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:56.662166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:56.662168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:56.662258 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:56.662619 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:56.662623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:56.662625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:56.662824 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:56.662936 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:56.663475 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 22:34:56.663621 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
WARNING: no policy specified for kudu/127.18.211.130@KRBTEST.COM; defaulting to no policy
I20260811 22:34:56.664686 21573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:56.650174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43547 (local address 127.18.211.129:60971)
0811 22:34:56.650627 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:56.650670 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:56.651889 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:56.652578 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:56.652590 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:56.653129 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:56.653830 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:56.653847 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.654835 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.654841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:56.655747 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:56.655758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.656314 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.656319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:56.656506 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:56.657204 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:56.657275 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:56.659461 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:56.661734 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:56.661741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:56.661763 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:56.662047 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:56.662358 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:56.662361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:56.662363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:56.662516 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:56.662936 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:56.662945 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:56.663222 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 22:34:56.663433 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:34:56.663781 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":119,"threads_started":1}
I20260811 22:34:56.666055 21570 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43547
I20260811 22:34:56.666316 21570 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:56.666827 21570 heartbeater.cc:511] Master 127.18.211.190:43547 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.130@KRBTEST.COM" created.
I20260811 22:34:56.668437 21228 ts_manager.cc:194] Registered new tserver with Master: 35862fdc92dd4f589b44e54dc18054ee (127.18.211.129:41535)
I20260811 22:34:56.669654 21228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.129@KRBTEST.COM'} at 127.18.211.129:60971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
Entry for principal kudu/127.18.211.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab.
I20260811 22:34:56.684945 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43547
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:56.800406 21577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:56.800694 21577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:56.800738 21577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:56.804826 21577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:56.804895 21577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:56.804980 21577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:56.810582 21577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.130.keytab
--principal=kudu/127.18.211.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:56.811697 21577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:56.812769 21577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:56.822577 21593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:56.822608 21589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:56.822602 21590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:56.822860 21577 server_base.cc:1034] running on GCE node
I20260811 22:34:56.823437 21577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:56.824081 21577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:56.825345 21577 hybrid_clock.cc:648] HybridClock initialized: now 1786487696825321 us; error 48 us; skew 500 ppm
Aug 11 22:34:56 dist-test-slave-8hhm krb5kdc[21142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487696, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:56.828316 21577 init.cc:377] Logged in from keytab as kudu/127.18.211.130@KRBTEST.COM (short username kudu)
I20260811 22:34:56.829276 21577 webserver.cc:460] Webserver started at http://127.18.211.130:37131/ using document root <none> and password file <none>
I20260811 22:34:56.829869 21577 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:56.829970 21577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:56.830215 21577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:56.832515 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "442e3fa63bf1402e882a75a951ac5122"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "c4551265c727e5f91d3e11394ca3e519"
server_key_iv: "c308d50a8a4db6bb210790e80c64ead2"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.833094 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "442e3fa63bf1402e882a75a951ac5122"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "c4551265c727e5f91d3e11394ca3e519"
server_key_iv: "c308d50a8a4db6bb210790e80c64ead2"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.836867 21577 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:34:56.839411 21603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:56.840752 21577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:56.840924 21577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "442e3fa63bf1402e882a75a951ac5122"
format_stamp: "Formatted at 2026-08-11 22:34:56 on dist-test-slave-8hhm"
server_key: "c4551265c727e5f91d3e11394ca3e519"
server_key_iv: "c308d50a8a4db6bb210790e80c64ead2"
server_key_version: "encryptionkey@0"
I20260811 22:34:56.841050 21577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:56.852579 21577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:56.855428 21577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:56.855641 21577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:56.856230 21577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:56.857064 21577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:56.857134 21577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:56.857203 21577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:56.857265 21577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:56.866252 21577 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:37677
I20260811 22:34:56.866282 21795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:37677 every 8 connection(s)
I20260811 22:34:56.867161 21577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:34:56.872419 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21577
I20260811 22:34:56.872550 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:34:56.872854 19278 external_mini_cluster.cc:1428] Setting key ee7f384fed0dcfd337143b136689cf33
Aug 11 22:34:56 dist-test-slave-8hhm krb5kdc[21142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487696, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.130@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:56.883684 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:56.868966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:42313 (local address 127.18.211.190:43547)
0811 22:34:56.869118 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:56.869124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:56.870005 (+ 881us) server_negotiation.cc:407] Connection header received
0811 22:34:56.871384 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:56.871387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:56.871446 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:56.871560 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:56.872877 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.873600 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.875246 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.875505 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.875631 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:56.878529 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:56.878536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:56.878542 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:56.878587 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:56.880427 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:56.881102 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:56.881145 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:56.881148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:56.881299 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:56.881911 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:56.881914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:56.881916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:56.882272 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:56.882383 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:56.883243 (+ 860us) server_negotiation.cc:299] Negotiation successful
0811 22:34:56.883436 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:34:56.884986 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:56.869364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43547 (local address 127.18.211.130:42313)
0811 22:34:56.869850 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:56.869894 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:56.871161 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:56.871748 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:56.871755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:56.872124 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:56.872732 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:56.872748 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.873740 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.873744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:56.875114 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:56.875130 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:56.875707 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:56.875714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:56.875978 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:56.876639 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:56.876669 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:56.878344 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:56.880550 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:56.880556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:56.880570 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:56.880990 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:56.881461 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:56.881467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:56.881469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:56.881710 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:56.882421 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:56.882428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:56.882861 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 22:34:56.883134 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 22:34:56.883516 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.18.211.131@KRBTEST.COM; defaulting to no policy
I20260811 22:34:56.886436 21796 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43547
I20260811 22:34:56.886754 21796 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:56.887413 21796 heartbeater.cc:511] Master 127.18.211.190:43547 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.131@KRBTEST.COM" created.
I20260811 22:34:56.888850 21228 ts_manager.cc:194] Registered new tserver with Master: 442e3fa63bf1402e882a75a951ac5122 (127.18.211.130:37677)
I20260811 22:34:56.889552 21228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.130@KRBTEST.COM'} at 127.18.211.130:42313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
Entry for principal kudu/127.18.211.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab.
I20260811 22:34:56.906461 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43547
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:34:57.023447 21808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:57.023722 21808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:57.023764 21808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:57.027783 21808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:57.027845 21808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:57.027938 21808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:57.033281 21808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.131.keytab
--principal=kudu/127.18.211.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:43547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:57.034353 21808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:57.035395 21808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:57.045570 21816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:57.045611 21819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:57.045570 21817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:57.045712 21808 server_base.cc:1034] running on GCE node
I20260811 22:34:57.046339 21808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:57.047212 21808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:57.048375 21808 hybrid_clock.cc:648] HybridClock initialized: now 1786487697048344 us; error 49 us; skew 500 ppm
Aug 11 22:34:57 dist-test-slave-8hhm krb5kdc[21142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:34:57.052394 21808 init.cc:377] Logged in from keytab as kudu/127.18.211.131@KRBTEST.COM (short username kudu)
I20260811 22:34:57.053462 21808 webserver.cc:460] Webserver started at http://127.18.211.131:33151/ using document root <none> and password file <none>
I20260811 22:34:57.054189 21808 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:57.054287 21808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:57.054535 21808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:57.056929 21808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "b58673d1b9a54226874d76aef7e25e2b"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "6f8d1c483f27acc5879e32650fa43b11"
server_key_iv: "9f9ce12f4273232179c306dc0112ac9a"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.057725 21808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "b58673d1b9a54226874d76aef7e25e2b"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "6f8d1c483f27acc5879e32650fa43b11"
server_key_iv: "9f9ce12f4273232179c306dc0112ac9a"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.062665 21808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:34:57.065786 21835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:57.067248 21808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:34:57.067385 21808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "b58673d1b9a54226874d76aef7e25e2b"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "6f8d1c483f27acc5879e32650fa43b11"
server_key_iv: "9f9ce12f4273232179c306dc0112ac9a"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.067540 21808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:57.091722 21808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:57.094841 21808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:57.095048 21808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:57.095661 21808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:57.096515 21808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:57.096570 21808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:57.096609 21808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:57.096644 21808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:57.105717 21808 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:39545
I20260811 22:34:57.105747 22029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:39545 every 8 connection(s)
I20260811 22:34:57.106666 21808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:57.115022 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 21808
I20260811 22:34:57.115231 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:34:57.115603 19278 external_mini_cluster.cc:1428] Setting key 45a73662150d86efadb4184f258e113b
Aug 11 22:34:57 dist-test-slave-8hhm krb5kdc[21142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.131@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:34:57.121309 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.108715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:49577 (local address 127.18.211.190:43547)
0811 22:34:57.108909 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.108912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.110012 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 22:34:57.111438 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.111441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.111490 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.111566 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:57.112779 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.113396 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.114554 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.114767 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.114844 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.117785 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:57.117788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:57.117790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:57.117814 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:57.119189 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:57.119735 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:57.119743 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:57.119744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:57.119824 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:57.120178 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:57.120182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:57.120183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:57.120371 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:57.120489 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.120989 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.121113 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:34:57.122159 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.109294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:43547 (local address 127.18.211.131:49577)
0811 22:34:57.109855 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.109897 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:57.111236 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:57.111775 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:57.111782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:57.112151 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:57.112645 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.112660 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.113515 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.113518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.114442 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.114452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.114925 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.114930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.115156 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.115895 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:34:57.115915 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:57.117603 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:57.119322 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:57.119328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:57.119344 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:57.119620 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:57.119918 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:34:57.119921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:34:57.119923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:34:57.120079 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:34:57.120487 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:57.120494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:57.120753 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 22:34:57.120947 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 22:34:57.121340 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":180,"threads_started":1}
I20260811 22:34:57.123252 22030 heartbeater.cc:347] Connected to a master server at 127.18.211.190:43547
I20260811 22:34:57.123509 22030 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:57.124022 22030 heartbeater.cc:511] Master 127.18.211.190:43547 requested a full tablet report, sending...
I20260811 22:34:57.125088 21228 ts_manager.cc:194] Registered new tserver with Master: b58673d1b9a54226874d76aef7e25e2b (127.18.211.131:39545)
I20260811 22:34:57.125679 21228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.211.131@KRBTEST.COM'} at 127.18.211.131:49577
I20260811 22:34:57.129670 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:34:57.139087 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.131942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53940 (local address 127.18.211.190:43547)
0811 22:34:57.132096 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.132100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.132243 (+ 143us) server_negotiation.cc:407] Connection header received
0811 22:34:57.132312 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.132314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.132353 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.132419 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:57.133141 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.133706 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.134492 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.134695 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.134771 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.136075 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:57.136078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:57.136080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:34:57.136102 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:57.137597 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:57.137968 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:57.137970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:57.137971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:57.138074 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:34:57.138345 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:34:57.138347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:34:57.138348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:34:57.138550 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:57.138655 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.138861 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.138952 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:34:57.142019 21228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:34:57.145300 21228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:34:57.160852 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.154055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41535 (local address 127.0.0.1:41106)
0811 22:34:57.154455 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.154515 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:57.154658 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:57.155926 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:57.155932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:57.155955 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:57.156114 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.156124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.157754 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.157757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.159188 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.159199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.160276 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.160283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.160394 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.160431 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:34:57.160546 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:34:57.160667 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":180,"spinlock_wait_cycles":5760,"thread_start_us":109,"threads_started":1}
I20260811 22:34:57.160990 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.154917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.18.211.131:39545)
0811 22:34:57.155703 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.155713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.155733 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:34:57.155794 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.155798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.155996 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.156167 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:57.156416 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.157576 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.159623 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.160427 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.160546 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.160660 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.160754 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.160846 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":136,"threads_started":1}
I20260811 22:34:57.161060 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.154071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:39545 (local address 127.0.0.1:41326)
0811 22:34:57.154455 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.154515 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:57.154658 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:57.156158 (+ 1500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:57.156164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:57.156187 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:57.156297 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.156304 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.157742 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.157751 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.159486 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.159498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.160534 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.160538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.160623 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.160646 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:34:57.160724 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:34:57.160802 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":227,"spinlock_wait_cycles":14464,"thread_start_us":125,"threads_started":1}
I20260811 22:34:57.161494 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.154969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.18.211.129:41535)
0811 22:34:57.155423 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.155442 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.155469 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:34:57.155552 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.155557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.155760 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.155889 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:57.156255 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.157576 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.159322 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.160135 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.160288 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.160433 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.161236 (+ 803us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.161350 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":83,"threads_started":1}
I20260811 22:34:57.161629 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.155259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.18.211.130:37677)
0811 22:34:57.155577 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.155587 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.156317 (+ 730us) server_negotiation.cc:407] Connection header received
0811 22:34:57.156402 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.156407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.156552 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.156821 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:57.157012 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.157922 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.159665 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.160254 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.160368 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.160495 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.160886 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.161464 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":132,"threads_started":1}
I20260811 22:34:57.162590 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.154865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:37677 (local address 127.0.0.1:35058)
0811 22:34:57.156111 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.156128 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:57.156222 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:57.156798 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:57.156801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:57.156821 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:57.156910 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.156916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.158069 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.158075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.159516 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.159524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.160559 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.160562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.160653 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.160674 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:34:57.160743 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:34:57.160802 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1163,"mutex_wait_us":308,"thread_start_us":103,"threads_started":1}
I20260811 22:34:57.164511 21453 tablet_service.cc:1467] Processing CreateTablet for tablet d8d675863bc34ae9bc903fc0918224ee (DEFAULT_TABLE table=test-table [id=7e4db8ae03b94e699b72c17af1a4c6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:57.164619 21917 tablet_service.cc:1467] Processing CreateTablet for tablet d8d675863bc34ae9bc903fc0918224ee (DEFAULT_TABLE table=test-table [id=7e4db8ae03b94e699b72c17af1a4c6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:57.165143 21685 tablet_service.cc:1467] Processing CreateTablet for tablet d8d675863bc34ae9bc903fc0918224ee (DEFAULT_TABLE table=test-table [id=7e4db8ae03b94e699b72c17af1a4c6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:57.165752 21453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d675863bc34ae9bc903fc0918224ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:57.165748 21917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d675863bc34ae9bc903fc0918224ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:57.166066 21685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d675863bc34ae9bc903fc0918224ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:57.172119 22061 tablet_bootstrap.cc:492] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee: Bootstrap starting.
I20260811 22:34:57.173635 22062 tablet_bootstrap.cc:492] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Bootstrap starting.
I20260811 22:34:57.174026 22063 tablet_bootstrap.cc:492] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Bootstrap starting.
I20260811 22:34:57.176158 22062 tablet_bootstrap.cc:654] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:57.176265 22063 tablet_bootstrap.cc:654] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:57.176452 22061 tablet_bootstrap.cc:654] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:57.177368 22062 log.cc:824] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:57.177430 22061 log.cc:824] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:57.177445 22063 log.cc:824] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:57.179221 22061 tablet_bootstrap.cc:492] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee: No bootstrap required, opened a new log
I20260811 22:34:57.179358 22062 tablet_bootstrap.cc:492] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: No bootstrap required, opened a new log
I20260811 22:34:57.179358 22063 tablet_bootstrap.cc:492] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: No bootstrap required, opened a new log
I20260811 22:34:57.179477 22061 ts_tablet_manager.cc:1397] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 22:34:57.179545 22062 ts_tablet_manager.cc:1397] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:34:57.179545 22063 ts_tablet_manager.cc:1397] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:34:57.183363 22061 raft_consensus.cc:348] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.183635 22061 raft_consensus.cc:374] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:57.183460 22062 raft_consensus.cc:348] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.183460 22063 raft_consensus.cc:348] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.183692 22061 raft_consensus.cc:729] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35862fdc92dd4f589b44e54dc18054ee, State: Initialized, Role: FOLLOWER
I20260811 22:34:57.183704 22062 raft_consensus.cc:374] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:57.183704 22063 raft_consensus.cc:374] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:57.183766 22062 raft_consensus.cc:729] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58673d1b9a54226874d76aef7e25e2b, State: Initialized, Role: FOLLOWER
I20260811 22:34:57.183775 22063 raft_consensus.cc:729] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442e3fa63bf1402e882a75a951ac5122, State: Initialized, Role: FOLLOWER
I20260811 22:34:57.184155 22061 consensus_queue.cc:260] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.184202 22062 consensus_queue.cc:260] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.184202 22063 consensus_queue.cc:260] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.185149 21796 heartbeater.cc:503] Master 127.18.211.190:43547 was elected leader, sending a full tablet report...
I20260811 22:34:57.185237 22063 ts_tablet_manager.cc:1428] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:34:57.185237 22062 ts_tablet_manager.cc:1428] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:34:57.185343 22061 ts_tablet_manager.cc:1428] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:34:57.185640 21570 heartbeater.cc:503] Master 127.18.211.190:43547 was elected leader, sending a full tablet report...
I20260811 22:34:57.185969 22030 heartbeater.cc:503] Master 127.18.211.190:43547 was elected leader, sending a full tablet report...
I20260811 22:34:57.238902 22078 raft_consensus.cc:482] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:57.239109 22078 raft_consensus.cc:504] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.240257 22078 leader_election.cc:288] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b58673d1b9a54226874d76aef7e25e2b (127.18.211.131:39545), 442e3fa63bf1402e882a75a951ac5122 (127.18.211.130:37677)
I20260811 22:34:57.244446 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.240599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:36805 (local address 127.18.211.130:37677)
0811 22:34:57.240729 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.240733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.241080 (+ 347us) server_negotiation.cc:407] Connection header received
0811 22:34:57.241194 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.241198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.241299 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.241364 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:57.241748 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.242321 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.243403 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.243827 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.243923 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.243965 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.244223 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.244319 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:34:57.244503 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.240512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:37677 (local address 127.18.211.129:36805)
0811 22:34:57.240966 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.240979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:57.241094 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:57.241410 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:57.241413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:57.241434 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:57.241583 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.241590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.242445 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.242447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.243252 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.243262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.243960 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.243966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.244085 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.244113 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:34:57.244195 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:34:57.244270 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":60,"thread_start_us":118,"threads_started":1}
I20260811 22:34:57.244541 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.240616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:43153 (local address 127.18.211.131:39545)
0811 22:34:57.240740 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.240745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.241004 (+ 259us) server_negotiation.cc:407] Connection header received
0811 22:34:57.241498 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.241500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.241543 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.241632 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:57.241914 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.242392 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.243474 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.243942 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.244110 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.244168 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.244344 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.244424 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:34:57.244509 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.240512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:39545 (local address 127.18.211.129:43153)
0811 22:34:57.240902 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.240915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:57.241068 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:57.241653 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:57.241657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:57.241676 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:57.241803 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.241808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.242504 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.242510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.243297 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.243306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.244105 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.244108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.244209 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.244235 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:34:57.244312 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:34:57.244376 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":159,"threads_started":1}
I20260811 22:34:57.245299 21717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d675863bc34ae9bc903fc0918224ee" candidate_uuid: "35862fdc92dd4f589b44e54dc18054ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442e3fa63bf1402e882a75a951ac5122" is_pre_election: true
I20260811 22:34:57.245649 21955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d675863bc34ae9bc903fc0918224ee" candidate_uuid: "35862fdc92dd4f589b44e54dc18054ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58673d1b9a54226874d76aef7e25e2b" is_pre_election: true
I20260811 22:34:57.245697 21717 raft_consensus.cc:2454] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35862fdc92dd4f589b44e54dc18054ee in term 0.
I20260811 22:34:57.245942 21955 raft_consensus.cc:2454] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35862fdc92dd4f589b44e54dc18054ee in term 0.
I20260811 22:34:57.246212 21368 leader_election.cc:302] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35862fdc92dd4f589b44e54dc18054ee, 442e3fa63bf1402e882a75a951ac5122; no voters:
I20260811 22:34:57.246541 22078 raft_consensus.cc:2781] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:34:57.246647 22078 raft_consensus.cc:482] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:34:57.246707 22078 raft_consensus.cc:3037] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:57.247766 22078 raft_consensus.cc:504] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.248138 22078 leader_election.cc:288] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [CANDIDATE]: Term 1 election: Requested vote from peers b58673d1b9a54226874d76aef7e25e2b (127.18.211.131:39545), 442e3fa63bf1402e882a75a951ac5122 (127.18.211.130:37677)
I20260811 22:34:57.248513 21955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d675863bc34ae9bc903fc0918224ee" candidate_uuid: "35862fdc92dd4f589b44e54dc18054ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58673d1b9a54226874d76aef7e25e2b"
I20260811 22:34:57.248513 21717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d675863bc34ae9bc903fc0918224ee" candidate_uuid: "35862fdc92dd4f589b44e54dc18054ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442e3fa63bf1402e882a75a951ac5122"
I20260811 22:34:57.248651 21717 raft_consensus.cc:3037] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:57.248641 21955 raft_consensus.cc:3037] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:57.250010 21955 raft_consensus.cc:2454] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35862fdc92dd4f589b44e54dc18054ee in term 1.
I20260811 22:34:57.250010 21717 raft_consensus.cc:2454] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35862fdc92dd4f589b44e54dc18054ee in term 1.
I20260811 22:34:57.250319 21368 leader_election.cc:302] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 35862fdc92dd4f589b44e54dc18054ee, 442e3fa63bf1402e882a75a951ac5122, b58673d1b9a54226874d76aef7e25e2b; no voters:
I20260811 22:34:57.250461 22078 raft_consensus.cc:2781] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:57.250682 22078 raft_consensus.cc:686] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 1 LEADER]: Becoming Leader. State: Replica: 35862fdc92dd4f589b44e54dc18054ee, State: Running, Role: LEADER
I20260811 22:34:57.251036 22078 consensus_queue.cc:237] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } }
I20260811 22:34:57.254565 21228 catalog_manager.cc:5495] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 35862fdc92dd4f589b44e54dc18054ee (127.18.211.129). New cstate: current_term: 1 leader_uuid: "35862fdc92dd4f589b44e54dc18054ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35862fdc92dd4f589b44e54dc18054ee" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 41535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 } health_report { overall_health: UNKNOWN } } }
I20260811 22:34:57.286868 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.283267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.18.211.129:41535)
0811 22:34:57.283437 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.283441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.283459 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:34:57.283532 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.283535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.283595 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.283711 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:57.283983 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.284490 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.285372 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.285594 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.285703 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.286036 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:57.286463 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:34:57.286600 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.286664 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.286723 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:34:57.292897 21955 raft_consensus.cc:1261] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 1 FOLLOWER]: Refusing update from remote peer 35862fdc92dd4f589b44e54dc18054ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:57.292897 21717 raft_consensus.cc:1261] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 1 FOLLOWER]: Refusing update from remote peer 35862fdc92dd4f589b44e54dc18054ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:57.293695 22078 consensus_queue.cc:1035] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "442e3fa63bf1402e882a75a951ac5122" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 37677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:57.293924 22082 consensus_queue.cc:1035] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "b58673d1b9a54226874d76aef7e25e2b" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 39545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:57.301450 22091 mvcc.cc:205] Tried to move back new op lower bound from 7317453608104161280 to 7317453607943634944. Current Snapshot: MvccSnapshot[applied={T|T < 7317453608104161280}]
I20260811 22:34:57.310601 21223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:57.310810 21223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:57.313576 21223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79334718c0d5406886d41da508c960af: Sending DeleteTablet for 3 replicas of tablet d8d675863bc34ae9bc903fc0918224ee
I20260811 22:34:57.314486 21917 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d675863bc34ae9bc903fc0918224ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:57 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:41326
I20260811 22:34:57.314570 21685 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d675863bc34ae9bc903fc0918224ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:57 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:35058
I20260811 22:34:57.314556 21453 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d675863bc34ae9bc903fc0918224ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:57 UTC) from {username='kudu', principal='kudu/127.18.211.190@KRBTEST.COM'} at 127.0.0.1:41106
I20260811 22:34:57.315099 22101 tablet_replica.cc:330] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: stopping tablet replica
I20260811 22:34:57.315330 22105 tablet_replica.cc:330] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee: stopping tablet replica
I20260811 22:34:57.315466 22101 raft_consensus.cc:2229] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:57.315717 22105 raft_consensus.cc:2229] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:34:57.315846 22101 raft_consensus.cc:2258] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:57.315938 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21334
I20260811 22:34:57.316290 22105 raft_consensus.cc:2258] T d8d675863bc34ae9bc903fc0918224ee P 35862fdc92dd4f589b44e54dc18054ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:57.318086 22101 ts_tablet_manager.cc:1905] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:57.319687 22104 tablet_replica.cc:330] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: stopping tablet replica
I20260811 22:34:57.320081 22104 raft_consensus.cc:2229] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:57.320436 22104 raft_consensus.cc:2258] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:57.321244 22101 ts_tablet_manager.cc:1918] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:57.321357 22101 log.cc:1196] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/d8d675863bc34ae9bc903fc0918224ee
I20260811 22:34:57.321717 22101 ts_tablet_manager.cc:1939] T d8d675863bc34ae9bc903fc0918224ee P b58673d1b9a54226874d76aef7e25e2b: Deleting consensus metadata
I20260811 22:34:57.322189 22104 ts_tablet_manager.cc:1905] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:57.324725 22104 ts_tablet_manager.cc:1918] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:57.324867 22104 log.cc:1196] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/d8d675863bc34ae9bc903fc0918224ee
I20260811 22:34:57.325276 21196 catalog_manager.cc:4841] TS b58673d1b9a54226874d76aef7e25e2b (127.18.211.131:39545): tablet d8d675863bc34ae9bc903fc0918224ee (table test-table [id=7e4db8ae03b94e699b72c17af1a4c6f3]) successfully deleted
I20260811 22:34:57.325438 22104 ts_tablet_manager.cc:1939] T d8d675863bc34ae9bc903fc0918224ee P 442e3fa63bf1402e882a75a951ac5122: Deleting consensus metadata
I20260811 22:34:57.326705 21200 catalog_manager.cc:4841] TS 442e3fa63bf1402e882a75a951ac5122 (127.18.211.130:37677): tablet d8d675863bc34ae9bc903fc0918224ee (table test-table [id=7e4db8ae03b94e699b72c17af1a4c6f3]) successfully deleted
W20260811 22:34:57.331199 21200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:41535 (error 108)
I20260811 22:34:57.331524 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21577
I20260811 22:34:57.339771 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.337654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:41535 (local address 127.0.0.1:41126)
0811 22:34:57.339539 (+ 1885us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.339641 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1746}
W20260811 22:34:57.345165 21200 catalog_manager.cc:4584] TS 35862fdc92dd4f589b44e54dc18054ee (127.18.211.129:41535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8d675863bc34ae9bc903fc0918224ee: Network error: Client connection negotiation failed: client connection to 127.18.211.129:41535: connect: Connection refused (error 111)
I20260811 22:34:57.353871 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21808
I20260811 22:34:57.376219 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 21168
2026-08-11T22:34:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2660 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T22:34:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:34:57Z Disabled control of system clock
I20260811 22:34:57.420369 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:36709
--webserver_interface=127.18.211.190
--webserver_port=0
--builtin_ntp_servers=127.18.211.148:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:34:57.535604 22115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:57.535879 22115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:57.535913 22115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:57.539953 22115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:34:57.540007 22115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:57.540021 22115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:57.540053 22115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:34:57.540068 22115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:34:57.544661 22115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:36709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:57.545729 22115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:57.547199 22115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:57.556165 22125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:57.556195 22122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:57.556250 22123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:57.556475 22115 server_base.cc:1034] running on GCE node
I20260811 22:34:57.557119 22115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:57.558269 22115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:57.559440 22115 hybrid_clock.cc:648] HybridClock initialized: now 1786487697559410 us; error 41 us; skew 500 ppm
I20260811 22:34:57.561422 22115 webserver.cc:460] Webserver started at http://127.18.211.190:39591/ using document root <none> and password file <none>
I20260811 22:34:57.562028 22115 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:57.562093 22115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:57.562341 22115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:57.564476 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "b79d8ba582204c67b0f770705a012d5f"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "ddd120a31a578d18e4b3827f82b4708a"
server_key_iv: "24d3c7ffcc4855dbd5797d64afc230f1"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.565091 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "b79d8ba582204c67b0f770705a012d5f"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "ddd120a31a578d18e4b3827f82b4708a"
server_key_iv: "24d3c7ffcc4855dbd5797d64afc230f1"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.568770 22115 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:34:57.571144 22134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:57.572348 22115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:34:57.572448 22115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "b79d8ba582204c67b0f770705a012d5f"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "ddd120a31a578d18e4b3827f82b4708a"
server_key_iv: "24d3c7ffcc4855dbd5797d64afc230f1"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.572546 22115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:57.592939 22115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:57.593657 22115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:57.593811 22115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:57.600912 22115 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:36709
I20260811 22:34:57.600948 22221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:36709 every 8 connection(s)
I20260811 22:34:57.601931 22115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:34:57.605100 22222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:57.607525 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22115
I20260811 22:34:57.607676 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:34:57.607968 19278 external_mini_cluster.cc:1428] Setting key f7fb0a89307da732ce99a855a89e5aa0
I20260811 22:34:57.612543 22222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f: Bootstrap starting.
I20260811 22:34:57.615626 22222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:57.616672 22222 log.cc:824] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:57.619414 22222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f: No bootstrap required, opened a new log
I20260811 22:34:57.620042 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.609522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.18.211.190:36709)
0811 22:34:57.610415 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.610441 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.610504 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:34:57.611490 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.611519 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.612079 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.612514 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:57.613020 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.614722 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.615678 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.616467 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.616676 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.617720 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:57.617728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:57.617746 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:57.618317 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:57.618397 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:57.618430 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:57.618569 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.618701 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.618996 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":110,"threads_started":1}
I20260811 22:34:57.624042 22222 raft_consensus.cc:348] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79d8ba582204c67b0f770705a012d5f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36709 } }
I20260811 22:34:57.624298 22222 raft_consensus.cc:374] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:57.624339 22222 raft_consensus.cc:729] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79d8ba582204c67b0f770705a012d5f, State: Initialized, Role: FOLLOWER
I20260811 22:34:57.624801 22222 consensus_queue.cc:260] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79d8ba582204c67b0f770705a012d5f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36709 } }
I20260811 22:34:57.624956 22222 raft_consensus.cc:388] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:34:57.625015 22222 raft_consensus.cc:482] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:34:57.625092 22222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:57.626127 22222 raft_consensus.cc:504] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79d8ba582204c67b0f770705a012d5f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36709 } }
I20260811 22:34:57.626452 22222 leader_election.cc:302] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b79d8ba582204c67b0f770705a012d5f; no voters:
I20260811 22:34:57.626780 22222 leader_election.cc:288] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:34:57.626863 22230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:57.627184 22230 raft_consensus.cc:686] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [term 1 LEADER]: Becoming Leader. State: Replica: b79d8ba582204c67b0f770705a012d5f, State: Running, Role: LEADER
I20260811 22:34:57.627504 22230 consensus_queue.cc:237] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79d8ba582204c67b0f770705a012d5f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36709 } }
I20260811 22:34:57.628263 22222 sys_catalog.cc:564] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:34:57.629230 22232 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b79d8ba582204c67b0f770705a012d5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79d8ba582204c67b0f770705a012d5f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36709 } } }
I20260811 22:34:57.629365 22232 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:57.629297 22231 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b79d8ba582204c67b0f770705a012d5f. Latest consensus state: current_term: 1 leader_uuid: "b79d8ba582204c67b0f770705a012d5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79d8ba582204c67b0f770705a012d5f" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36709 } } }
I20260811 22:34:57.629565 22231 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f [sys.catalog]: This master's current role is: LEADER
I20260811 22:34:57.629774 22242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:34:57.632997 22242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:34:57.638646 22242 catalog_manager.cc:1347] Generated new cluster ID: d6b0b335169248a0bc703b7ff67d340c
I20260811 22:34:57.638722 22242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:34:57.656271 22242 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:34:57.657198 22242 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:34:57.662256 22242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f: Generated new TSK 0
I20260811 22:34:57.662961 22242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:34:57.666608 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.129:0
--local_ip_for_outbound_sockets=127.18.211.129
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:36709
--builtin_ntp_servers=127.18.211.148:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:34:57.797399 22265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:57.797670 22265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:57.797712 22265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:57.802002 22265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:57.802062 22265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:57.802151 22265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.129
I20260811 22:34:57.806931 22265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.211.129
--webserver_port=0
--tserver_master_addrs=127.18.211.190:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.129
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:57.808028 22265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:57.809540 22265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:57.819072 22276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:57.819039 22275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:57.819058 22281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:57.819403 22265 server_base.cc:1034] running on GCE node
I20260811 22:34:57.819798 22265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:57.820430 22265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:57.821593 22265 hybrid_clock.cc:648] HybridClock initialized: now 1786487697821573 us; error 32 us; skew 500 ppm
I20260811 22:34:57.823419 22265 webserver.cc:460] Webserver started at http://127.18.211.129:38125/ using document root <none> and password file <none>
I20260811 22:34:57.824043 22265 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:57.824136 22265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:57.824376 22265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:57.826541 22265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data/instance:
uuid: "5f81b047a20f45a697a1063b2e1d8f72"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "2156e85bf1e9f178bf1fd857d6b52a8c"
server_key_iv: "6c15d22be97ba174138076c754f4eae4"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.827131 22265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance:
uuid: "5f81b047a20f45a697a1063b2e1d8f72"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "2156e85bf1e9f178bf1fd857d6b52a8c"
server_key_iv: "6c15d22be97ba174138076c754f4eae4"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.831127 22265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:34:57.833707 22288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:57.834928 22265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:34:57.835109 22265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/wal
uuid: "5f81b047a20f45a697a1063b2e1d8f72"
format_stamp: "Formatted at 2026-08-11 22:34:57 on dist-test-slave-8hhm"
server_key: "2156e85bf1e9f178bf1fd857d6b52a8c"
server_key_iv: "6c15d22be97ba174138076c754f4eae4"
server_key_version: "encryptionkey@0"
I20260811 22:34:57.835240 22265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:57.857676 22265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:57.858409 22265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:57.858618 22265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:57.859253 22265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:57.860246 22265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:57.860332 22265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:57.860396 22265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:57.860440 22265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:57.869488 22265 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.129:42319
I20260811 22:34:57.869532 22480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.129:42319 every 8 connection(s)
I20260811 22:34:57.870402 22265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/data/info.pb
I20260811 22:34:57.875344 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22265
I20260811 22:34:57.875478 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-0/wal/instance
I20260811 22:34:57.875815 19278 external_mini_cluster.cc:1428] Setting key 0b7cc271dbc3db529535f27dfc9f00a6
I20260811 22:34:57.877770 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.130:0
--local_ip_for_outbound_sockets=127.18.211.130
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:36709
--builtin_ntp_servers=127.18.211.148:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:34:57.880120 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.872201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.129:40081 (local address 127.18.211.190:36709)
0811 22:34:57.872340 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:57.872344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:57.873467 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 22:34:57.874869 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:57.874872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:57.874918 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:57.874994 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:57.876151 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.876791 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.878108 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.878330 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.878446 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.879610 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:57.879613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:57.879616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:57.879627 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:57.879647 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:57.879655 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:57.879746 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:57.879918 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 22:34:57.879980 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:34:57.881145 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:57.872610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:36709 (local address 127.18.211.129:40081)
0811 22:34:57.873316 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:57.873358 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:57.874678 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:57.875167 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:57.875174 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:57.875270 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:57.876003 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.876019 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.876926 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.876929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.877976 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:57.877994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:57.878487 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:57.878510 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:57.878680 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:57.879334 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:57.879337 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:57.879381 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:57.879445 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:57.879748 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:57.879761 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:57.879770 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:34:57.879914 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:34:57.880201 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":119,"threads_started":1}
I20260811 22:34:57.882401 22481 heartbeater.cc:347] Connected to a master server at 127.18.211.190:36709
I20260811 22:34:57.882648 22481 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:57.883216 22481 heartbeater.cc:511] Master 127.18.211.190:36709 requested a full tablet report, sending...
I20260811 22:34:57.884704 22162 ts_manager.cc:194] Registered new tserver with Master: 5f81b047a20f45a697a1063b2e1d8f72 (127.18.211.129:42319)
I20260811 22:34:57.885883 22162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.129:40081
W20260811 22:34:57.995500 22486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:57.995819 22486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:57.995893 22486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:57.999902 22486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:57.999993 22486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:58.000085 22486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.130
I20260811 22:34:58.004787 22486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.211.130
--webserver_port=0
--tserver_master_addrs=127.18.211.190:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.130
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:58.005910 22486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:58.007346 22486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:58.017084 22495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:58.017098 22494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:58.017153 22497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:58.017508 22486 server_base.cc:1034] running on GCE node
I20260811 22:34:58.017881 22486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:58.018520 22486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:58.019723 22486 hybrid_clock.cc:648] HybridClock initialized: now 1786487698019717 us; error 36 us; skew 500 ppm
I20260811 22:34:58.021826 22486 webserver.cc:460] Webserver started at http://127.18.211.130:40529/ using document root <none> and password file <none>
I20260811 22:34:58.022423 22486 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:58.022529 22486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:58.022771 22486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:58.024875 22486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data/instance:
uuid: "d82da593b3bd40dbb46b2058e4b18c08"
format_stamp: "Formatted at 2026-08-11 22:34:58 on dist-test-slave-8hhm"
server_key: "ddf5bc62d7831d5b595c519a9219ffce"
server_key_iv: "24768a7a221f6aa4a2072df9b2dde87d"
server_key_version: "encryptionkey@0"
I20260811 22:34:58.025511 22486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance:
uuid: "d82da593b3bd40dbb46b2058e4b18c08"
format_stamp: "Formatted at 2026-08-11 22:34:58 on dist-test-slave-8hhm"
server_key: "ddf5bc62d7831d5b595c519a9219ffce"
server_key_iv: "24768a7a221f6aa4a2072df9b2dde87d"
server_key_version: "encryptionkey@0"
I20260811 22:34:58.029471 22486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:34:58.032173 22508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:58.033622 22486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:34:58.033766 22486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal
uuid: "d82da593b3bd40dbb46b2058e4b18c08"
format_stamp: "Formatted at 2026-08-11 22:34:58 on dist-test-slave-8hhm"
server_key: "ddf5bc62d7831d5b595c519a9219ffce"
server_key_iv: "24768a7a221f6aa4a2072df9b2dde87d"
server_key_version: "encryptionkey@0"
I20260811 22:34:58.033891 22486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:58.047130 22486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:58.047871 22486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:58.048098 22486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:58.048756 22486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:58.049880 22486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:58.049959 22486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:58.050033 22486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:58.050082 22486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:58.059691 22486 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.130:42855
I20260811 22:34:58.059711 22711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.130:42855 every 8 connection(s)
I20260811 22:34:58.060722 22486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/data/info.pb
I20260811 22:34:58.066212 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22486
I20260811 22:34:58.066383 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal/instance
I20260811 22:34:58.066715 19278 external_mini_cluster.cc:1428] Setting key f7df9648fda9377173767bb0b833d5e4
I20260811 22:34:58.068769 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.211.131:0
--local_ip_for_outbound_sockets=127.18.211.131
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:36709
--builtin_ntp_servers=127.18.211.148:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:34:58.071808 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.062729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:59977 (local address 127.18.211.190:36709)
0811 22:34:58.062902 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.062907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.064133 (+ 1226us) server_negotiation.cc:407] Connection header received
0811 22:34:58.065980 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.065985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.066051 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.066161 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:58.067498 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.068049 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.069509 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.069925 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.070046 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.071310 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:58.071313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:58.071315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:58.071326 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:58.071345 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:58.071353 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:58.071415 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.071633 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.071685 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:34:58.073140 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.063213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:36709 (local address 127.18.211.130:59977)
0811 22:34:58.063958 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.064013 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:58.065707 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:58.066376 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:58.066384 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:58.066521 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:58.067176 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.067198 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.068160 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.068166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.069359 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.069370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.070096 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.070099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.070249 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.070971 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:58.070975 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:58.071044 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:58.071142 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:58.071449 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:58.071463 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:58.071474 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:34:58.071632 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 22:34:58.071987 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":130,"threads_started":1}
I20260811 22:34:58.074491 22712 heartbeater.cc:347] Connected to a master server at 127.18.211.190:36709
I20260811 22:34:58.074752 22712 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:58.075376 22712 heartbeater.cc:511] Master 127.18.211.190:36709 requested a full tablet report, sending...
I20260811 22:34:58.076531 22162 ts_manager.cc:194] Registered new tserver with Master: d82da593b3bd40dbb46b2058e4b18c08 (127.18.211.130:42855)
I20260811 22:34:58.077188 22162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.130:59977
W20260811 22:34:58.190322 22718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:34:58.190573 22718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:34:58.190610 22718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:34:58.194566 22718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:34:58.194624 22718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:34:58.194674 22718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.211.131
I20260811 22:34:58.199244 22718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.211.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.211.131
--webserver_port=0
--tserver_master_addrs=127.18.211.190:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.211.131
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:34:58.200261 22718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:34:58.201665 22718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:34:58.211447 22735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:58.211478 22727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:34:58.211447 22728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:34:58.211808 22718 server_base.cc:1034] running on GCE node
I20260811 22:34:58.212188 22718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:34:58.212816 22718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:34:58.213976 22718 hybrid_clock.cc:648] HybridClock initialized: now 1786487698213959 us; error 33 us; skew 500 ppm
I20260811 22:34:58.215881 22718 webserver.cc:460] Webserver started at http://127.18.211.131:39813/ using document root <none> and password file <none>
I20260811 22:34:58.216473 22718 fs_manager.cc:359] Metadata directory not provided
I20260811 22:34:58.216537 22718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:34:58.216747 22718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:34:58.218941 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data/instance:
uuid: "4cb54acd3a7e4d658d45b153a1662fd5"
format_stamp: "Formatted at 2026-08-11 22:34:58 on dist-test-slave-8hhm"
server_key: "61a546ca68a8e7d67f2605485231b854"
server_key_iv: "875d10bd3155b3047b4930fd61f49287"
server_key_version: "encryptionkey@0"
I20260811 22:34:58.219542 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance:
uuid: "4cb54acd3a7e4d658d45b153a1662fd5"
format_stamp: "Formatted at 2026-08-11 22:34:58 on dist-test-slave-8hhm"
server_key: "61a546ca68a8e7d67f2605485231b854"
server_key_iv: "875d10bd3155b3047b4930fd61f49287"
server_key_version: "encryptionkey@0"
I20260811 22:34:58.223428 22718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:34:58.225976 22741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:58.227233 22718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:34:58.227380 22718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal
uuid: "4cb54acd3a7e4d658d45b153a1662fd5"
format_stamp: "Formatted at 2026-08-11 22:34:58 on dist-test-slave-8hhm"
server_key: "61a546ca68a8e7d67f2605485231b854"
server_key_iv: "875d10bd3155b3047b4930fd61f49287"
server_key_version: "encryptionkey@0"
I20260811 22:34:58.227489 22718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:34:58.245919 22718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:34:58.246640 22718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:34:58.246845 22718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:34:58.247462 22718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:34:58.248484 22718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:34:58.248562 22718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:58.248636 22718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:34:58.248692 22718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:34:58.257983 22718 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.131:34961
I20260811 22:34:58.258021 22939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.131:34961 every 8 connection(s)
I20260811 22:34:58.258953 22718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/data/info.pb
I20260811 22:34:58.268224 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 22718
I20260811 22:34:58.268357 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal/instance
I20260811 22:34:58.268491 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.260904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.131:33341 (local address 127.18.211.190:36709)
0811 22:34:58.261104 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.261108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.262202 (+ 1094us) server_negotiation.cc:407] Connection header received
0811 22:34:58.263576 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.263578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.263624 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.263720 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:58.264604 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.265145 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.266447 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.266703 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.266824 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.267961 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:58.267965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:58.267974 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:58.267988 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:58.268007 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:58.268015 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:58.268092 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.268281 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.268347 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:34:58.268702 19278 external_mini_cluster.cc:1428] Setting key 4b8f6ce04282cdfc550c2f62781b927e
I20260811 22:34:58.269531 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.261387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:36709 (local address 127.18.211.131:33341)
0811 22:34:58.262053 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.262096 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:58.263385 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:58.263874 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:58.263881 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:58.263963 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:34:58.264476 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.264489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.265333 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.265336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.266249 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.266264 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.266879 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.266883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.267054 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.267683 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:34:58.267685 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:34:58.267730 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:34:58.267795 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:34:58.268107 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:34:58.268120 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:34:58.268129 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:34:58.268279 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 22:34:58.268565 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":105,"threads_started":1}
I20260811 22:34:58.270704 22940 heartbeater.cc:347] Connected to a master server at 127.18.211.190:36709
I20260811 22:34:58.270954 22940 heartbeater.cc:464] Registering TS with master...
I20260811 22:34:58.271504 22940 heartbeater.cc:511] Master 127.18.211.190:36709 requested a full tablet report, sending...
I20260811 22:34:58.272532 22161 ts_manager.cc:194] Registered new tserver with Master: 4cb54acd3a7e4d658d45b153a1662fd5 (127.18.211.131:34961)
I20260811 22:34:58.273082 22161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.211.131:33341
I20260811 22:34:58.282848 19278 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:34:58.288534 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.284837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.18.211.190:36709)
0811 22:34:58.284982 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.284986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.285159 (+ 173us) server_negotiation.cc:407] Connection header received
0811 22:34:58.285320 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.285322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.285363 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.285428 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:34:58.285689 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.286205 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.287030 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.287264 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.287340 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.288088 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:34:58.288091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:34:58.288093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:34:58.288104 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:34:58.288121 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:34:58.288129 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:34:58.288187 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.288283 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.288332 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:34:58.291530 22161 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:34:58.294734 22161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:34:58.309098 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.303418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52940 (local address 127.18.211.131:34961)
0811 22:34:58.303927 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.303941 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.304222 (+ 281us) server_negotiation.cc:407] Connection header received
0811 22:34:58.304317 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.304323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.304551 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.304727 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:58.305121 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.306142 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.307575 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.308359 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.308489 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.308676 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.308834 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.308932 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":166,"threads_started":1}
I20260811 22:34:58.309093 22961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.303210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:34961 (local address 127.0.0.1:52940)
0811 22:34:58.303862 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.303918 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:58.304066 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:58.304825 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:58.304831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:58.304857 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:58.305000 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.305006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.306277 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.306282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.307430 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.307441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.308488 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.308492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.308606 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.308645 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:34:58.308800 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:34:58.308930 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260811 22:34:58.309829 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.303592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.130:42855 (local address 127.0.0.1:45310)
0811 22:34:58.304655 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.304668 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:58.304754 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:58.305298 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:58.305300 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:58.305321 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:58.305398 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.305403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.306815 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.306818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.308072 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.308083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.309311 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.309315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.309492 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.309531 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:34:58.309608 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:34:58.309701 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":576,"thread_start_us":123,"threads_started":1}
I20260811 22:34:58.309963 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.304241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.18.211.129:42319)
0811 22:34:58.305061 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.305072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.305093 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:34:58.305166 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.305170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.305375 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.305531 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:58.305742 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.306842 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.308123 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.308907 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.309044 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.309241 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.309702 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.309794 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":145,"threads_started":1}
I20260811 22:34:58.310328 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.303993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:42319 (local address 127.0.0.1:54800)
0811 22:34:58.304298 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.304313 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:58.304413 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:58.305542 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:58.305544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:58.305560 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:58.305626 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.305631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.306952 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.306958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.307987 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.308004 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.309311 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.309315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.309507 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.309548 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:34:58.309608 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 22:34:58.310176 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":533,"thread_start_us":95,"threads_started":1}
I20260811 22:34:58.310671 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.303832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.18.211.130:42855)
0811 22:34:58.304697 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.304709 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.304732 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:34:58.304922 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.304928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.305137 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.305323 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:58.305505 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.306685 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.308178 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.308956 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.309071 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.309241 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.310260 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.310362 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":134,"threads_started":1}
I20260811 22:34:58.312672 22828 tablet_service.cc:1467] Processing CreateTablet for tablet 6f21dcca2f494373b0de07ff8a7db108 (DEFAULT_TABLE table=test-table [id=024f3451ec1d43149743549cf54528a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:58.313177 22591 tablet_service.cc:1467] Processing CreateTablet for tablet 6f21dcca2f494373b0de07ff8a7db108 (DEFAULT_TABLE table=test-table [id=024f3451ec1d43149743549cf54528a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:58.313916 22828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f21dcca2f494373b0de07ff8a7db108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:58.314205 22591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f21dcca2f494373b0de07ff8a7db108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:58.314165 22373 tablet_service.cc:1467] Processing CreateTablet for tablet 6f21dcca2f494373b0de07ff8a7db108 (DEFAULT_TABLE table=test-table [id=024f3451ec1d43149743549cf54528a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:34:58.315171 22373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f21dcca2f494373b0de07ff8a7db108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:34:58.320389 22971 tablet_bootstrap.cc:492] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Bootstrap starting.
I20260811 22:34:58.321942 22980 tablet_bootstrap.cc:492] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Bootstrap starting.
I20260811 22:34:58.322758 22971 tablet_bootstrap.cc:654] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:58.322863 22981 tablet_bootstrap.cc:492] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: Bootstrap starting.
I20260811 22:34:58.323561 22971 log.cc:824] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:58.324151 22980 tablet_bootstrap.cc:654] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:58.324878 22981 tablet_bootstrap.cc:654] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: Neither blocks nor log segments found. Creating new log.
I20260811 22:34:58.325008 22980 log.cc:824] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:58.325700 22981 log.cc:824] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: Log is configured to *not* fsync() on all Append() calls
I20260811 22:34:58.326233 22971 tablet_bootstrap.cc:492] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: No bootstrap required, opened a new log
I20260811 22:34:58.326442 22971 ts_tablet_manager.cc:1397] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:34:58.327105 22980 tablet_bootstrap.cc:492] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: No bootstrap required, opened a new log
I20260811 22:34:58.327293 22980 ts_tablet_manager.cc:1397] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:34:58.327705 22981 tablet_bootstrap.cc:492] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: No bootstrap required, opened a new log
I20260811 22:34:58.327904 22981 ts_tablet_manager.cc:1397] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:34:58.330150 22971 raft_consensus.cc:348] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.330389 22971 raft_consensus.cc:374] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:58.330443 22971 raft_consensus.cc:729] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d82da593b3bd40dbb46b2058e4b18c08, State: Initialized, Role: FOLLOWER
I20260811 22:34:58.330892 22971 consensus_queue.cc:260] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.331611 22712 heartbeater.cc:503] Master 127.18.211.190:36709 was elected leader, sending a full tablet report...
I20260811 22:34:58.331823 22980 raft_consensus.cc:348] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.332077 22980 raft_consensus.cc:374] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:58.331971 22981 raft_consensus.cc:348] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.332134 22980 raft_consensus.cc:729] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb54acd3a7e4d658d45b153a1662fd5, State: Initialized, Role: FOLLOWER
I20260811 22:34:58.332185 22981 raft_consensus.cc:374] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:34:58.332244 22981 raft_consensus.cc:729] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f81b047a20f45a697a1063b2e1d8f72, State: Initialized, Role: FOLLOWER
I20260811 22:34:58.332549 22980 consensus_queue.cc:260] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.332602 22981 consensus_queue.cc:260] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.332957 22971 ts_tablet_manager.cc:1428] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 22:34:58.333566 22981 ts_tablet_manager.cc:1428] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260811 22:34:58.333590 22980 ts_tablet_manager.cc:1428] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:34:58.333761 22940 heartbeater.cc:503] Master 127.18.211.190:36709 was elected leader, sending a full tablet report...
I20260811 22:34:58.333796 22481 heartbeater.cc:503] Master 127.18.211.190:36709 was elected leader, sending a full tablet report...
W20260811 22:34:58.371620 22482 tablet.cc:2369] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:34:58.443692 22985 raft_consensus.cc:482] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:34:58.443953 22985 raft_consensus.cc:504] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.445276 22985 leader_election.cc:288] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cb54acd3a7e4d658d45b153a1662fd5 (127.18.211.131:34961), 5f81b047a20f45a697a1063b2e1d8f72 (127.18.211.129:42319)
I20260811 22:34:58.449467 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.445641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:47963 (local address 127.18.211.131:34961)
0811 22:34:58.445803 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.445808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.445826 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:34:58.445914 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.445917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.445964 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.446056 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:58.446577 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.447183 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.448377 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.448814 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.448912 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.448953 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.449253 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.449337 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:34:58.449469 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.445503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.131:34961 (local address 127.18.211.130:47963)
0811 22:34:58.445659 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.445692 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:58.445810 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:58.446207 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:58.446209 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:58.446226 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:58.446374 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.446382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.447299 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.447302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.448231 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.448242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.448949 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.448956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.449072 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.449099 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:34:58.449177 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:34:58.449316 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":40}
I20260811 22:34:58.449497 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.445969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.211.130:40761 (local address 127.18.211.129:42319)
0811 22:34:58.446128 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.446133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.446305 (+ 172us) server_negotiation.cc:407] Connection header received
0811 22:34:58.446387 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.446391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.446465 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.446559 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:34:58.446847 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.447375 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.448445 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.448903 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.449034 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.449084 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.449316 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.449388 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:34:58.449492 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.445851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:42319 (local address 127.18.211.130:40761)
0811 22:34:58.446208 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.446220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:34:58.446311 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:34:58.446591 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:34:58.446594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:34:58.446613 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:34:58.446727 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.446733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.447485 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.447487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.448278 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:34:58.448287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.449057 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.449060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:34:58.449135 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.449154 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:34:58.449296 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:34:58.449360 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":19,"thread_start_us":136,"threads_started":1}
I20260811 22:34:58.450169 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f21dcca2f494373b0de07ff8a7db108" candidate_uuid: "d82da593b3bd40dbb46b2058e4b18c08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f81b047a20f45a697a1063b2e1d8f72" is_pre_election: true
I20260811 22:34:58.450186 22859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f21dcca2f494373b0de07ff8a7db108" candidate_uuid: "d82da593b3bd40dbb46b2058e4b18c08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" is_pre_election: true
I20260811 22:34:58.450485 22859 raft_consensus.cc:2454] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d82da593b3bd40dbb46b2058e4b18c08 in term 0.
I20260811 22:34:58.450498 22405 raft_consensus.cc:2454] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d82da593b3bd40dbb46b2058e4b18c08 in term 0.
I20260811 22:34:58.451037 22517 leader_election.cc:302] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f81b047a20f45a697a1063b2e1d8f72, d82da593b3bd40dbb46b2058e4b18c08; no voters:
I20260811 22:34:58.451332 22985 raft_consensus.cc:2781] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:34:58.451413 22985 raft_consensus.cc:482] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:34:58.451443 22985 raft_consensus.cc:3037] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:58.452786 22985 raft_consensus.cc:504] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.453140 22985 leader_election.cc:288] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [CANDIDATE]: Term 1 election: Requested vote from peers 4cb54acd3a7e4d658d45b153a1662fd5 (127.18.211.131:34961), 5f81b047a20f45a697a1063b2e1d8f72 (127.18.211.129:42319)
I20260811 22:34:58.453552 22859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f21dcca2f494373b0de07ff8a7db108" candidate_uuid: "d82da593b3bd40dbb46b2058e4b18c08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb54acd3a7e4d658d45b153a1662fd5"
I20260811 22:34:58.453610 22405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f21dcca2f494373b0de07ff8a7db108" candidate_uuid: "d82da593b3bd40dbb46b2058e4b18c08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f81b047a20f45a697a1063b2e1d8f72"
I20260811 22:34:58.453706 22859 raft_consensus.cc:3037] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:58.453735 22405 raft_consensus.cc:3037] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:34:58.454725 22405 raft_consensus.cc:2454] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d82da593b3bd40dbb46b2058e4b18c08 in term 1.
I20260811 22:34:58.454741 22859 raft_consensus.cc:2454] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d82da593b3bd40dbb46b2058e4b18c08 in term 1.
I20260811 22:34:58.455052 22517 leader_election.cc:302] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cb54acd3a7e4d658d45b153a1662fd5, d82da593b3bd40dbb46b2058e4b18c08; no voters:
I20260811 22:34:58.455287 22985 raft_consensus.cc:2781] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:34:58.455561 22985 raft_consensus.cc:686] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 1 LEADER]: Becoming Leader. State: Replica: d82da593b3bd40dbb46b2058e4b18c08, State: Running, Role: LEADER
I20260811 22:34:58.455919 22985 consensus_queue.cc:237] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } }
I20260811 22:34:58.459235 22160 catalog_manager.cc:5495] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 reported cstate change: term changed from 0 to 1, leader changed from <none> to d82da593b3bd40dbb46b2058e4b18c08 (127.18.211.130). New cstate: current_term: 1 leader_uuid: "d82da593b3bd40dbb46b2058e4b18c08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d82da593b3bd40dbb46b2058e4b18c08" member_type: VOTER last_known_addr { host: "127.18.211.130" port: 42855 } health_report { overall_health: HEALTHY } } }
I20260811 22:34:58.498458 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.494712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.18.211.130:42855)
0811 22:34:58.494862 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:34:58.494865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:34:58.494878 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:34:58.494959 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:34:58.494961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:34:58.495018 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:34:58.495114 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:34:58.495413 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.495904 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.496806 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:34:58.497012 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:34:58.497088 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:34:58.497441 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:34:58.497911 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:34:58.498012 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:34:58.498174 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 22:34:58.498238 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:34:58.504730 22859 raft_consensus.cc:1261] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 1 FOLLOWER]: Refusing update from remote peer d82da593b3bd40dbb46b2058e4b18c08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:58.504734 22405 raft_consensus.cc:1261] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 1 FOLLOWER]: Refusing update from remote peer d82da593b3bd40dbb46b2058e4b18c08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:34:58.505548 22985 consensus_queue.cc:1035] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cb54acd3a7e4d658d45b153a1662fd5" member_type: VOTER last_known_addr { host: "127.18.211.131" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:34:58.505967 22993 consensus_queue.cc:1035] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f81b047a20f45a697a1063b2e1d8f72" member_type: VOTER last_known_addr { host: "127.18.211.129" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:34:58.509895 22941 tablet.cc:2369] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:34:58.517828 23000 mvcc.cc:205] Tried to move back new op lower bound from 7317453613067620352 to 7317453612878458880. Current Snapshot: MvccSnapshot[applied={T|T < 7317453613067620352}]
I20260811 22:34:58.523238 22160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:58.523418 22160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:34:58.525895 22160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b79d8ba582204c67b0f770705a012d5f: Sending DeleteTablet for 3 replicas of tablet 6f21dcca2f494373b0de07ff8a7db108
I20260811 22:34:58.526782 22591 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f21dcca2f494373b0de07ff8a7db108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:58 UTC) from {username='slave'} at 127.0.0.1:45310
I20260811 22:34:58.526796 22828 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f21dcca2f494373b0de07ff8a7db108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:58 UTC) from {username='slave'} at 127.0.0.1:52940
I20260811 22:34:58.526796 22373 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f21dcca2f494373b0de07ff8a7db108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:34:58 UTC) from {username='slave'} at 127.0.0.1:54800
I20260811 22:34:58.527320 23011 tablet_replica.cc:330] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: stopping tablet replica
I20260811 22:34:58.527349 23015 tablet_replica.cc:330] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72: stopping tablet replica
I20260811 22:34:58.527683 23015 raft_consensus.cc:2229] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:58.527681 23011 raft_consensus.cc:2229] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:34:58.527997 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22265
I20260811 22:34:58.528031 23015 raft_consensus.cc:2258] T 6f21dcca2f494373b0de07ff8a7db108 P 5f81b047a20f45a697a1063b2e1d8f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:58.528043 23011 raft_consensus.cc:2258] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:58.528844 23012 tablet_replica.cc:330] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: stopping tablet replica
I20260811 22:34:58.531755 23012 raft_consensus.cc:2229] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:34:58.532150 23012 raft_consensus.cc:2258] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:34:58.532770 23011 ts_tablet_manager.cc:1905] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:58.533685 23012 ts_tablet_manager.cc:1905] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:34:58.535960 23011 ts_tablet_manager.cc:1918] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:58.536151 23011 log.cc:1196] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-2/wal/wals/6f21dcca2f494373b0de07ff8a7db108
I20260811 22:34:58.536469 23012 ts_tablet_manager.cc:1918] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:34:58.536590 23012 log.cc:1196] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Deleting WAL directory at /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487433619239-19278-0/minicluster-data/ts-1/wal/wals/6f21dcca2f494373b0de07ff8a7db108
I20260811 22:34:58.536689 23011 ts_tablet_manager.cc:1939] T 6f21dcca2f494373b0de07ff8a7db108 P 4cb54acd3a7e4d658d45b153a1662fd5: Deleting consensus metadata
I20260811 22:34:58.536962 23012 ts_tablet_manager.cc:1939] T 6f21dcca2f494373b0de07ff8a7db108 P d82da593b3bd40dbb46b2058e4b18c08: Deleting consensus metadata
I20260811 22:34:58.538172 22135 catalog_manager.cc:4841] TS d82da593b3bd40dbb46b2058e4b18c08 (127.18.211.130:42855): tablet 6f21dcca2f494373b0de07ff8a7db108 (table test-table [id=024f3451ec1d43149743549cf54528a2]) successfully deleted
I20260811 22:34:58.538172 22138 catalog_manager.cc:4841] TS 4cb54acd3a7e4d658d45b153a1662fd5 (127.18.211.131:34961): tablet 6f21dcca2f494373b0de07ff8a7db108 (table test-table [id=024f3451ec1d43149743549cf54528a2]) successfully deleted
W20260811 22:34:58.549510 22138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.211.129:42319 (error 108)
I20260811 22:34:58.550104 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22486
I20260811 22:34:58.553462 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:34:58.550806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.129:42319 (local address 127.0.0.1:54814)
0811 22:34:58.553238 (+ 2432us) negotiation.cc:107] Waiting for socket to connect
0811 22:34:58.553337 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.129:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2273}
W20260811 22:34:58.553658 22138 catalog_manager.cc:4584] TS 5f81b047a20f45a697a1063b2e1d8f72 (127.18.211.129:42319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f21dcca2f494373b0de07ff8a7db108: Network error: Client connection negotiation failed: client connection to 127.18.211.129:42319: connect: Connection refused (error 111)
I20260811 22:34:58.562348 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22718
I20260811 22:34:58.589489 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 22115
2026-08-11T22:34:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1216 ms)
[----------] 6 tests from EncryptionPolicyTest (11630 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:34:58 dist-test-slave-8hhm krb5kdc[23025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:34:58 dist-test-slave-8hhm krb5kdc[23025](info): set up 2 sockets
Aug 11 22:34:58 dist-test-slave-8hhm krb5kdc[23025](info): commencing operation
krb5kdc: starting...
W20260811 22:34:59.903416 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:34:59 dist-test-slave-8hhm krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:00.040481 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:00.156598 23051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:00.156925 23051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:00.157004 23051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:00.161018 23051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:00.161141 23051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:00.161182 23051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:00.161252 23051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:00.161283 23051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:00.166690 23051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:00.167819 23051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:00.168898 23051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:00.178103 23061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:00.178138 23059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:00.178103 23064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:00.178511 23051 server_base.cc:1034] running on GCE node
I20260811 22:35:00.179116 23051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:00.180238 23051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:00.181422 23051 hybrid_clock.cc:648] HybridClock initialized: now 1786487700181397 us; error 40 us; skew 500 ppm
Aug 11 22:35:00 dist-test-slave-8hhm krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487700, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:00.188652 23051 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:00.189702 23051 webserver.cc:460] Webserver started at http://127.0.0.1:38813/ using document root <none> and password file <none>
I20260811 22:35:00.190263 23051 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:00.190323 23051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:00.190503 23051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:00.192556 23051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca"
format_stamp: "Formatted at 2026-08-11 22:35:00 on dist-test-slave-8hhm"
server_key: "66b62d97d0e852c15cfebd290216f874"
server_key_iv: "68acb01e9fb5fadad4e6d871e16c88f5"
server_key_version: "encryptionkey@0"
I20260811 22:35:00.193094 23051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca"
format_stamp: "Formatted at 2026-08-11 22:35:00 on dist-test-slave-8hhm"
server_key: "66b62d97d0e852c15cfebd290216f874"
server_key_iv: "68acb01e9fb5fadad4e6d871e16c88f5"
server_key_version: "encryptionkey@0"
I20260811 22:35:00.196818 23051 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:35:00.199358 23079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:00.200585 23051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 22:35:00.200744 23051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca"
format_stamp: "Formatted at 2026-08-11 22:35:00 on dist-test-slave-8hhm"
server_key: "66b62d97d0e852c15cfebd290216f874"
server_key_iv: "68acb01e9fb5fadad4e6d871e16c88f5"
server_key_version: "encryptionkey@0"
I20260811 22:35:00.200851 23051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:00.214125 23051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:00.216917 23051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:00.217139 23051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:00.224435 23051 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35069
I20260811 22:35:00.224448 23178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35069 every 8 connection(s)
I20260811 22:35:00.225513 23051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:00.228376 23184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:00.228468 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23051
I20260811 22:35:00.228662 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:00.228969 19278 external_mini_cluster.cc:1428] Setting key 4c9c07bdfac278eb76d49703283cd25e
I20260811 22:35:00.235404 23184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca: Bootstrap starting.
I20260811 22:35:00.238328 23184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca: Neither blocks nor log segments found. Creating new log.
Aug 11 22:35:00 dist-test-slave-8hhm krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:35:00.239324 23184 log.cc:824] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:00.241926 23184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca: No bootstrap required, opened a new log
I20260811 22:35:00.244472 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:00.230319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51817 (local address 127.0.0.1:35069)
0811 22:35:00.230742 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:00.230757 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:00.230792 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:35:00.231665 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:00.231684 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:00.232025 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:00.232311 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:00.233307 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:00.234292 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:00.235480 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:00.235981 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:00.236108 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:00.239330 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:00.239339 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:00.239359 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:00.239403 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:00.241551 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:00.242094 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:00.242099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:00.242103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:00.242214 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:00.242575 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:00.242580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:00.242586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:00.242981 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:00.243214 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:00.243543 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 22:35:00.243762 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":146,"threads_started":1}
I20260811 22:35:00.245967 23184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35069 } }
I20260811 22:35:00.246223 23184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:00.246313 23184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cfb0c8208064ef7a0d3d6d8aee8c2ca, State: Initialized, Role: FOLLOWER
I20260811 22:35:00.246824 23184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35069 } }
I20260811 22:35:00.246994 23184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:00.247097 23184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:00.247198 23184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:00.247812 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:00.248283 23184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35069 } }
I20260811 22:35:00.248638 23184 leader_election.cc:302] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cfb0c8208064ef7a0d3d6d8aee8c2ca; no voters:
I20260811 22:35:00.249027 23184 leader_election.cc:288] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:00.249293 23196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:00.249691 23196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [term 1 LEADER]: Becoming Leader. State: Replica: 0cfb0c8208064ef7a0d3d6d8aee8c2ca, State: Running, Role: LEADER
I20260811 22:35:00.250030 23196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35069 } }
I20260811 22:35:00.250443 23184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:00.251884 23200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cfb0c8208064ef7a0d3d6d8aee8c2ca. Latest consensus state: current_term: 1 leader_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35069 } } }
I20260811 22:35:00.252019 23200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:00.252260 23198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfb0c8208064ef7a0d3d6d8aee8c2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35069 } } }
I20260811 22:35:00.252429 23198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:00.252465 23213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:00.255851 23213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:00.260927 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:00.250433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41831 (local address 127.0.0.1:35069)
0811 22:35:00.250542 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:00.250546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:00.250670 (+ 124us) server_negotiation.cc:407] Connection header received
0811 22:35:00.250797 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:00.250801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:00.250846 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:00.250932 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:00.253262 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:00.254011 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:00.255158 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:00.255477 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:00.255573 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:00.257443 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:00.257447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:00.257450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:00.257474 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:00.259017 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:00.259509 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:00.259512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:00.259514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:00.259602 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:00.259932 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:00.259935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:00.259937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:00.260122 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:00.260280 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:00.260644 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 22:35:00.260751 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:35:00.263062 23213 catalog_manager.cc:1347] Generated new cluster ID: c682c2a87b77402d92c280535b0ca0de
I20260811 22:35:00.263177 23213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:00.277943 23213 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:00.279160 23213 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:00.289942 23213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cfb0c8208064ef7a0d3d6d8aee8c2ca: Generated new TSK 0
I20260811 22:35:00.290735 23213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:00.303054 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23051
W20260811 22:35:00.318743 23019 connection.cc:535] client connection to 127.0.0.1:35069 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35069): Connection reset by peer (error 104)
2026-08-11T22:35:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1706 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:00 dist-test-slave-8hhm krb5kdc[23238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:00 dist-test-slave-8hhm krb5kdc[23238](info): set up 2 sockets
Aug 11 22:35:00 dist-test-slave-8hhm krb5kdc[23238](info): commencing operation
krb5kdc: starting...
W20260811 22:35:01.618861 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:01 dist-test-slave-8hhm krb5kdc[23238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:01.759958 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39685
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:01.877019 23262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:01.877331 23262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:01.877374 23262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:01.881407 23262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:01.881476 23262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:01.881496 23262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:01.881513 23262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:01.881529 23262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:01.886849 23262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39685
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:01.887948 23262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:01.888954 23262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:01.897876 23272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:01.897888 23271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:01.897890 23276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:01.898111 23262 server_base.cc:1034] running on GCE node
I20260811 22:35:01.899000 23262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:01.900169 23262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:01.901293 23262 hybrid_clock.cc:648] HybridClock initialized: now 1786487701901277 us; error 32 us; skew 500 ppm
Aug 11 22:35:01 dist-test-slave-8hhm krb5kdc[23238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487701, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:01.904280 23262 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:01.905385 23262 webserver.cc:460] Webserver started at http://127.0.0.1:34699/ using document root <none> and password file <none>
I20260811 22:35:01.905994 23262 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:01.906059 23262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:01.906329 23262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:01.908483 23262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "8011214614a6425192f714f15ed04fcb"
format_stamp: "Formatted at 2026-08-11 22:35:01 on dist-test-slave-8hhm"
server_key: "a78efd8ff9bd47abdd4d65e98c38ceb4"
server_key_iv: "86f6a0cf81ef23e7e6ee6d576a5ed98c"
server_key_version: "encryptionkey@0"
I20260811 22:35:01.909096 23262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "8011214614a6425192f714f15ed04fcb"
format_stamp: "Formatted at 2026-08-11 22:35:01 on dist-test-slave-8hhm"
server_key: "a78efd8ff9bd47abdd4d65e98c38ceb4"
server_key_iv: "86f6a0cf81ef23e7e6ee6d576a5ed98c"
server_key_version: "encryptionkey@0"
I20260811 22:35:01.912962 23262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:35:01.915539 23290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:01.916791 23262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:35:01.916945 23262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "8011214614a6425192f714f15ed04fcb"
format_stamp: "Formatted at 2026-08-11 22:35:01 on dist-test-slave-8hhm"
server_key: "a78efd8ff9bd47abdd4d65e98c38ceb4"
server_key_iv: "86f6a0cf81ef23e7e6ee6d576a5ed98c"
server_key_version: "encryptionkey@0"
I20260811 22:35:01.917068 23262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:01.931800 23262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:01.934996 23262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:01.935232 23262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:01.943157 23262 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39685
I20260811 22:35:01.943185 23397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39685 every 8 connection(s)
I20260811 22:35:01.944299 23262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:01.947201 23398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:01.947975 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23262
I20260811 22:35:01.948143 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:01.948493 19278 external_mini_cluster.cc:1428] Setting key 8da4d7a5d3976d81f7674fc3a612e49e
I20260811 22:35:01.953826 23398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb: Bootstrap starting.
I20260811 22:35:01.956410 23398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:01.957244 23398 log.cc:824] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:01.958367 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:01.950020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50961 (local address 127.0.0.1:39685)
0811 22:35:01.950521 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:01.950537 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:01.950575 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:35:01.951255 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:01.951273 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:01.951635 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:01.952016 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:01.952461 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:01.953536 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:01.954539 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:01.955065 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:01.955270 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:01.956271 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:01.956276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:01.956291 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:01.956753 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:01.956815 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:01.956840 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:01.956987 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:01.957187 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:35:01.957400 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":144,"threads_started":1}
I20260811 22:35:01.959514 23398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb: No bootstrap required, opened a new log
I20260811 22:35:01.961614 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:01.963719 23398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011214614a6425192f714f15ed04fcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 22:35:01.964080 23398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:01.964160 23398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8011214614a6425192f714f15ed04fcb, State: Initialized, Role: FOLLOWER
I20260811 22:35:01.964746 23398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011214614a6425192f714f15ed04fcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 22:35:01.964927 23398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:01.965022 23398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:01.965132 23398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:01.966441 23398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011214614a6425192f714f15ed04fcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 22:35:01.966856 23398 leader_election.cc:302] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8011214614a6425192f714f15ed04fcb; no voters:
I20260811 22:35:01.967237 23398 leader_election.cc:288] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:01.967315 23417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:01.967540 23417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [term 1 LEADER]: Becoming Leader. State: Replica: 8011214614a6425192f714f15ed04fcb, State: Running, Role: LEADER
I20260811 22:35:01.967921 23417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011214614a6425192f714f15ed04fcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } }
I20260811 22:35:01.968575 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:01.963887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55993 (local address 127.0.0.1:39685)
0811 22:35:01.964023 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:01.964027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:01.964194 (+ 167us) server_negotiation.cc:407] Connection header received
0811 22:35:01.964353 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:01.964356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:01.964403 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:01.964508 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:01.964820 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:01.965550 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:01.966687 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:01.966938 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:01.967116 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:01.968089 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:01.968093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:01.968096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:01.968110 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:01.968140 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:01.968161 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:01.968261 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:01.968393 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 22:35:01.968430 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:35:01.968701 23398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:01.969759 23418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8011214614a6425192f714f15ed04fcb. Latest consensus state: current_term: 1 leader_uuid: "8011214614a6425192f714f15ed04fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011214614a6425192f714f15ed04fcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } } }
I20260811 22:35:01.969905 23418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:01.970335 23430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:01.970852 23420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8011214614a6425192f714f15ed04fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011214614a6425192f714f15ed04fcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39685 } } }
I20260811 22:35:01.970955 23420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:01.973902 23430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:01.980655 23430 catalog_manager.cc:1347] Generated new cluster ID: a4e0a3b81bb64060b493880f10d2e4b5
I20260811 22:35:01.980741 23430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:01.989648 23430 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:01.990922 23430 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:01.999511 23430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8011214614a6425192f714f15ed04fcb: Generated new TSK 0
I20260811 22:35:02.000448 23430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:02.007838 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23262
W20260811 22:35:02.027252 23235 connection.cc:535] client connection to 127.0.0.1:39685 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39685): Connection reset by peer (error 104)
2026-08-11T22:35:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1708 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:02 dist-test-slave-8hhm krb5kdc[23448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:02 dist-test-slave-8hhm krb5kdc[23448](info): set up 2 sockets
Aug 11 22:35:02 dist-test-slave-8hhm krb5kdc[23448](info): commencing operation
krb5kdc: starting...
W20260811 22:35:03.316228 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:03 dist-test-slave-8hhm krb5kdc[23448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:03.454746 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:03.576808 23472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:03.577232 23472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:03.577334 23472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:03.581589 23472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:03.581698 23472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:03.581740 23472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:03.581779 23472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:03.581812 23472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:03.587489 23472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37461
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:03.588726 23472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:03.589908 23472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:03.599037 23485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:03.599180 23472 server_base.cc:1034] running on GCE node
W20260811 22:35:03.599031 23482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:03.599047 23492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:03.599900 23472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:03.601022 23472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:03.602195 23472 hybrid_clock.cc:648] HybridClock initialized: now 1786487703602182 us; error 33 us; skew 500 ppm
Aug 11 22:35:03 dist-test-slave-8hhm krb5kdc[23448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487703, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:03.605051 23472 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:03.606081 23472 webserver.cc:460] Webserver started at http://127.0.0.1:41235/ using document root <none> and password file <none>
I20260811 22:35:03.606678 23472 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:03.606779 23472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:03.607031 23472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:03.609131 23472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "8abf90d7c9fd435f8b74a6accf14cbb0"
format_stamp: "Formatted at 2026-08-11 22:35:03 on dist-test-slave-8hhm"
server_key: "1bd748bca55846b6943b8b52303717a5"
server_key_iv: "f59614812e1cf98bb1b49743cf99ca7f"
server_key_version: "encryptionkey@0"
I20260811 22:35:03.609786 23472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "8abf90d7c9fd435f8b74a6accf14cbb0"
format_stamp: "Formatted at 2026-08-11 22:35:03 on dist-test-slave-8hhm"
server_key: "1bd748bca55846b6943b8b52303717a5"
server_key_iv: "f59614812e1cf98bb1b49743cf99ca7f"
server_key_version: "encryptionkey@0"
I20260811 22:35:03.613651 23472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:35:03.616190 23508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:03.617517 23472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:35:03.617672 23472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "8abf90d7c9fd435f8b74a6accf14cbb0"
format_stamp: "Formatted at 2026-08-11 22:35:03 on dist-test-slave-8hhm"
server_key: "1bd748bca55846b6943b8b52303717a5"
server_key_iv: "f59614812e1cf98bb1b49743cf99ca7f"
server_key_version: "encryptionkey@0"
I20260811 22:35:03.617801 23472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:03.622624 23472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:03.625603 23472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:03.625968 23472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:03.633695 23472 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37461
I20260811 22:35:03.633716 23598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37461 every 8 connection(s)
I20260811 22:35:03.634718 23472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:03.637691 23599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:03.643265 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23472
I20260811 22:35:03.643419 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:03.643743 19278 external_mini_cluster.cc:1428] Setting key 31fd62968f726c9cbe11a1781a1d3d8f
I20260811 22:35:03.643878 23599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0: Bootstrap starting.
I20260811 22:35:03.646498 23599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:03.647267 23599 log.cc:824] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:03.649113 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:03.645190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35951 (local address 127.0.0.1:37461)
0811 22:35:03.645696 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:03.645712 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:03.645747 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:35:03.646420 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:03.646439 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:03.646777 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:03.647131 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:03.647914 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:03.647920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:03.647941 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:03.647951 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:03.648002 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:03.648021 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:03.648156 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:03.648266 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 22:35:03.648423 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":147,"threads_started":1}
I20260811 22:35:03.649415 23599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0: No bootstrap required, opened a new log
I20260811 22:35:03.652084 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:03.653671 23599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260811 22:35:03.653931 23599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:03.654012 23599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abf90d7c9fd435f8b74a6accf14cbb0, State: Initialized, Role: FOLLOWER
I20260811 22:35:03.654560 23599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260811 22:35:03.654727 23599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:03.654822 23599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:03.654925 23599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:03.655821 23599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260811 22:35:03.656173 23599 leader_election.cc:302] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8abf90d7c9fd435f8b74a6accf14cbb0; no voters:
I20260811 22:35:03.656514 23599 leader_election.cc:288] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:03.656652 23612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:03.656922 23612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [term 1 LEADER]: Becoming Leader. State: Replica: 8abf90d7c9fd435f8b74a6accf14cbb0, State: Running, Role: LEADER
I20260811 22:35:03.657305 23612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260811 22:35:03.657958 23599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:03.659135 23613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } } }
I20260811 22:35:03.659263 23613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:03.659569 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:03.654223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36193 (local address 127.0.0.1:37461)
0811 22:35:03.654356 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:03.654361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:03.657478 (+ 3117us) server_negotiation.cc:407] Connection header received
0811 22:35:03.657673 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:03.657676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:03.657723 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:03.657813 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:03.658739 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:03.658742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:03.658745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:03.658758 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:03.658787 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:03.658800 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:03.658878 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:03.659169 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:35:03.659209 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:35:03.659754 23629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:03.660262 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8abf90d7c9fd435f8b74a6accf14cbb0. Latest consensus state: current_term: 1 leader_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abf90d7c9fd435f8b74a6accf14cbb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } } }
I20260811 22:35:03.660401 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:03.662938 23629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:03.668577 23629 catalog_manager.cc:1347] Generated new cluster ID: d548eafe611743f7887f7d531b57c833
I20260811 22:35:03.668660 23629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:03.681106 23629 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:03.681484 23629 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:03.688657 23629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8abf90d7c9fd435f8b74a6accf14cbb0: Generated new TSK 0
I20260811 22:35:03.689484 23629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:03.693997 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23472
2026-08-11T22:35:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1682 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:03 dist-test-slave-8hhm krb5kdc[23650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:03 dist-test-slave-8hhm krb5kdc[23650](info): set up 2 sockets
Aug 11 22:35:03 dist-test-slave-8hhm krb5kdc[23650](info): commencing operation
krb5kdc: starting...
W20260811 22:35:04.998878 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:05 dist-test-slave-8hhm krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:05.138612 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:05.258479 23680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:05.258944 23680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:05.259032 23680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:05.263778 23680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:05.263862 23680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:05.263883 23680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:05.263900 23680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:05.263916 23680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:05.269644 23680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:05.270776 23680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:05.271839 23680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:05.280934 23691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:05.281095 23680 server_base.cc:1034] running on GCE node
W20260811 22:35:05.281102 23698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:05.281301 23692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:05.281939 23680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:05.283123 23680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:05.284343 23680 hybrid_clock.cc:648] HybridClock initialized: now 1786487705284308 us; error 56 us; skew 500 ppm
Aug 11 22:35:05 dist-test-slave-8hhm krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:05.290773 23680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:05.291811 23680 webserver.cc:460] Webserver started at http://127.0.0.1:42741/ using document root <none> and password file <none>
I20260811 22:35:05.292433 23680 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:05.292534 23680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:05.292791 23680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:05.295033 23680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "0963554497bb48f1bd4ab873f360309f"
format_stamp: "Formatted at 2026-08-11 22:35:05 on dist-test-slave-8hhm"
server_key: "4c7a31fc4f193850b316928f36e70f59"
server_key_iv: "d8c143116f18f43abcfe6143b1122f2a"
server_key_version: "encryptionkey@0"
I20260811 22:35:05.295637 23680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "0963554497bb48f1bd4ab873f360309f"
format_stamp: "Formatted at 2026-08-11 22:35:05 on dist-test-slave-8hhm"
server_key: "4c7a31fc4f193850b316928f36e70f59"
server_key_iv: "d8c143116f18f43abcfe6143b1122f2a"
server_key_version: "encryptionkey@0"
I20260811 22:35:05.299647 23680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 22:35:05.302433 23710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:05.303809 23680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:35:05.303972 23680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "0963554497bb48f1bd4ab873f360309f"
format_stamp: "Formatted at 2026-08-11 22:35:05 on dist-test-slave-8hhm"
server_key: "4c7a31fc4f193850b316928f36e70f59"
server_key_iv: "d8c143116f18f43abcfe6143b1122f2a"
server_key_version: "encryptionkey@0"
I20260811 22:35:05.304105 23680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:05.320351 23680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:05.327335 23680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:05.327570 23680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:05.337242 23680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38055
I20260811 22:35:05.337311 23804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38055 every 8 connection(s)
I20260811 22:35:05.338812 23680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:05.343215 23807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:05.347909 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23680
I20260811 22:35:05.348078 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:05.348388 19278 external_mini_cluster.cc:1428] Setting key 66501bd66533127a993cb8a51ccd2573
I20260811 22:35:05.350953 23807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f: Bootstrap starting.
I20260811 22:35:05.354480 23807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:05.355633 23807 log.cc:824] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:05.359182 23807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f: No bootstrap required, opened a new log
Aug 11 22:35:05 dist-test-slave-8hhm krb5kdc[23650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:35:05.363901 23807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0963554497bb48f1bd4ab873f360309f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260811 22:35:05.364228 23807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:05.364289 23807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0963554497bb48f1bd4ab873f360309f, State: Initialized, Role: FOLLOWER
I20260811 22:35:05.364830 23807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0963554497bb48f1bd4ab873f360309f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260811 22:35:05.364998 23807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:05.365068 23807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:05.365172 23807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:05.366590 23807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0963554497bb48f1bd4ab873f360309f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260811 22:35:05.367007 23807 leader_election.cc:302] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0963554497bb48f1bd4ab873f360309f; no voters:
I20260811 22:35:05.367360 23807 leader_election.cc:288] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:05.367437 23814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:05.367810 23814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [term 1 LEADER]: Becoming Leader. State: Replica: 0963554497bb48f1bd4ab873f360309f, State: Running, Role: LEADER
I20260811 22:35:05.368139 23814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0963554497bb48f1bd4ab873f360309f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260811 22:35:05.368752 23807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:05.369737 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:05.350615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42469 (local address 127.0.0.1:38055)
0811 22:35:05.351032 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:05.351053 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:05.351099 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:35:05.351780 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:05.351798 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:05.352155 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:05.352499 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:05.353445 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:05.355258 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:05.356678 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:05.357487 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:05.357616 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:05.362242 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:05.362252 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:05.362275 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:05.362321 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:05.364621 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:05.365447 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:05.365457 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:05.365466 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:05.365651 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:05.366054 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:05.366059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:05.366060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:05.366806 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:05.367846 (+ 1040us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:05.368095 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 22:35:05.368563 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":140,"threads_started":1}
I20260811 22:35:05.369879 23815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0963554497bb48f1bd4ab873f360309f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0963554497bb48f1bd4ab873f360309f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } } }
I20260811 22:35:05.370096 23815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:05.370321 23814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0963554497bb48f1bd4ab873f360309f. Latest consensus state: current_term: 1 leader_uuid: "0963554497bb48f1bd4ab873f360309f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0963554497bb48f1bd4ab873f360309f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } } }
I20260811 22:35:05.370410 23814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:05.373489 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 22:35:05.374650 23837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:05.374742 23837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:05.375471 23828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:05.376362 23828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:05.382529 23828 catalog_manager.cc:1347] Generated new cluster ID: 250a28377fcb41fa8b3b8e2160cb6722
I20260811 22:35:05.382616 23828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:05.388309 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:05.375804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36861 (local address 127.0.0.1:38055)
0811 22:35:05.375941 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:05.375946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:05.378813 (+ 2867us) server_negotiation.cc:407] Connection header received
0811 22:35:05.378868 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:05.378871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:05.378922 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:05.379014 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:05.379745 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:05.381013 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:05.382338 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:05.382824 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:05.382944 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:05.384652 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:05.384656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:05.384659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:05.384688 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:05.386508 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:05.386971 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:05.386975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:05.386977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:05.387101 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:05.387384 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:05.387388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:05.387389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:05.387626 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:05.387758 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:05.387998 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 22:35:05.388133 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:35:05.397305 23828 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:05.398650 23828 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:05.415385 23828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0963554497bb48f1bd4ab873f360309f: Generated new TSK 0
I20260811 22:35:05.416360 23828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:05.428303 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23680
2026-08-11T22:35:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1741 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:05 dist-test-slave-8hhm krb5kdc[23851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:05 dist-test-slave-8hhm krb5kdc[23851](info): set up 2 sockets
Aug 11 22:35:05 dist-test-slave-8hhm krb5kdc[23851](info): commencing operation
krb5kdc: starting...
W20260811 22:35:06.744940 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:06 dist-test-slave-8hhm krb5kdc[23851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:06.884433 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:07.006613 23879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:07.006945 23879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:07.007007 23879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:07.011061 23879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:07.011140 23879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:07.011160 23879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:07.011178 23879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:07.011193 23879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:07.016916 23879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:07.018172 23879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:07.019197 23879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:07.027946 23894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:07.027946 23895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:07.027949 23897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:07.028265 23879 server_base.cc:1034] running on GCE node
I20260811 22:35:07.028959 23879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:07.030211 23879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:07.031406 23879 hybrid_clock.cc:648] HybridClock initialized: now 1786487707031371 us; error 54 us; skew 500 ppm
Aug 11 22:35:07 dist-test-slave-8hhm krb5kdc[23851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:07.034521 23879 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:07.035518 23879 webserver.cc:460] Webserver started at http://127.0.0.1:40697/ using document root <none> and password file <none>
I20260811 22:35:07.036123 23879 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:07.036234 23879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:07.036479 23879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:07.038748 23879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "454c00be5e6740be8d56b49861554db3"
format_stamp: "Formatted at 2026-08-11 22:35:07 on dist-test-slave-8hhm"
server_key: "26ef1b4eba9bb6a4b6b160cf408fb97f"
server_key_iv: "8b9ac8955f39fe208a62ac1de33a2542"
server_key_version: "encryptionkey@0"
I20260811 22:35:07.039384 23879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "454c00be5e6740be8d56b49861554db3"
format_stamp: "Formatted at 2026-08-11 22:35:07 on dist-test-slave-8hhm"
server_key: "26ef1b4eba9bb6a4b6b160cf408fb97f"
server_key_iv: "8b9ac8955f39fe208a62ac1de33a2542"
server_key_version: "encryptionkey@0"
I20260811 22:35:07.043558 23879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 22:35:07.046271 23908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:07.047590 23879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:35:07.047756 23879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "454c00be5e6740be8d56b49861554db3"
format_stamp: "Formatted at 2026-08-11 22:35:07 on dist-test-slave-8hhm"
server_key: "26ef1b4eba9bb6a4b6b160cf408fb97f"
server_key_iv: "8b9ac8955f39fe208a62ac1de33a2542"
server_key_version: "encryptionkey@0"
I20260811 22:35:07.047888 23879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:07.066689 23879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:07.070647 23879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:07.070892 23879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:07.079056 23879 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34653
I20260811 22:35:07.079089 24008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34653 every 8 connection(s)
I20260811 22:35:07.080150 23879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:07.083371 24010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:07.084664 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 23879
I20260811 22:35:07.084820 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:07.085140 19278 external_mini_cluster.cc:1428] Setting key 0cc5316490b19c8e9c9b4ae56aa59355
I20260811 22:35:07.090184 24010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3: Bootstrap starting.
I20260811 22:35:07.092552 24010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:07.093412 24010 log.cc:824] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:07.095180 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:07.086763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45599 (local address 127.0.0.1:34653)
0811 22:35:07.087247 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:07.087264 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:07.087305 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:35:07.088098 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:07.088120 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:07.088501 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:07.088877 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:07.089400 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:07.090457 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:07.091406 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:07.091923 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:07.092107 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:07.093052 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:07.093058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:07.093072 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:07.093615 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:07.093683 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:07.093716 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:07.093827 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:07.093973 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 22:35:07.094196 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":146,"threads_started":1}
I20260811 22:35:07.095816 24010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3: No bootstrap required, opened a new log
I20260811 22:35:07.098232 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:07.099920 24010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454c00be5e6740be8d56b49861554db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 22:35:07.100237 24010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:07.100309 24010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454c00be5e6740be8d56b49861554db3, State: Initialized, Role: FOLLOWER
I20260811 22:35:07.100806 24010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454c00be5e6740be8d56b49861554db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 22:35:07.100946 24010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:07.101047 24010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:07.101188 24010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:07.102207 24010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454c00be5e6740be8d56b49861554db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 22:35:07.102568 24010 leader_election.cc:302] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 454c00be5e6740be8d56b49861554db3; no voters:
I20260811 22:35:07.102957 24010 leader_election.cc:288] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:07.103114 24028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:07.103338 24028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [term 1 LEADER]: Becoming Leader. State: Replica: 454c00be5e6740be8d56b49861554db3, State: Running, Role: LEADER
I20260811 22:35:07.103662 24028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454c00be5e6740be8d56b49861554db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 22:35:07.104498 24010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:07.104882 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:07.100400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55971 (local address 127.0.0.1:34653)
0811 22:35:07.100507 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:07.100513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:07.100725 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:35:07.100890 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:07.100893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:07.100949 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:07.101038 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:07.101365 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:07.101873 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:07.102810 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:07.103181 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:07.103289 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:07.104341 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:07.104344 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:07.104347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:07.104360 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:07.104391 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:07.104404 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:07.104488 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:07.104667 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 22:35:07.104725 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 22:35:07.105540 24029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "454c00be5e6740be8d56b49861554db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454c00be5e6740be8d56b49861554db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } } }
I20260811 22:35:07.105710 24029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:07.106022 24030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 454c00be5e6740be8d56b49861554db3. Latest consensus state: current_term: 1 leader_uuid: "454c00be5e6740be8d56b49861554db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454c00be5e6740be8d56b49861554db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } } }
I20260811 22:35:07.106120 24030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:07.106413 24043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:07.110186 24043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:07.116899 24043 catalog_manager.cc:1347] Generated new cluster ID: 5b7fe70ac2074ae99c28c23bdf0183c0
I20260811 22:35:07.117002 24043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:07.132493 24043 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:07.133663 24043 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:07.139514 24043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 454c00be5e6740be8d56b49861554db3: Generated new TSK 0
I20260811 22:35:07.140295 24043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:07.153419 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 23879
2026-08-11T22:35:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1729 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:07 dist-test-slave-8hhm krb5kdc[24058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:07 dist-test-slave-8hhm krb5kdc[24058](info): set up 2 sockets
Aug 11 22:35:07 dist-test-slave-8hhm krb5kdc[24058](info): commencing operation
krb5kdc: starting...
W20260811 22:35:08.462378 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:08 dist-test-slave-8hhm krb5kdc[24058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:08.598342 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:08.717561 24081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:08.717854 24081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:08.717897 24081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:08.722141 24081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:08.722218 24081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:08.722239 24081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:08.722255 24081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:08.722296 24081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:08.727855 24081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:08.729053 24081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:08.730252 24081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:08.739368 24092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:08.739346 24090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:08.739389 24095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:08.739909 24081 server_base.cc:1034] running on GCE node
I20260811 22:35:08.740520 24081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:08.741781 24081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:08.742980 24081 hybrid_clock.cc:648] HybridClock initialized: now 1786487708742959 us; error 34 us; skew 500 ppm
Aug 11 22:35:08 dist-test-slave-8hhm krb5kdc[24058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487708, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:08.745946 24081 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:08.747092 24081 webserver.cc:460] Webserver started at http://127.0.0.1:39455/ using document root <none> and password file <none>
I20260811 22:35:08.747717 24081 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:08.747813 24081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:08.748065 24081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:08.750340 24081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "dc17cfbc27fa453e8c73da1023af0f3f"
format_stamp: "Formatted at 2026-08-11 22:35:08 on dist-test-slave-8hhm"
server_key: "39d19a5e37448e71831b554918579daa"
server_key_iv: "2a94ece07b0ae1b819c4d4b5e669e413"
server_key_version: "encryptionkey@0"
I20260811 22:35:08.750945 24081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "dc17cfbc27fa453e8c73da1023af0f3f"
format_stamp: "Formatted at 2026-08-11 22:35:08 on dist-test-slave-8hhm"
server_key: "39d19a5e37448e71831b554918579daa"
server_key_iv: "2a94ece07b0ae1b819c4d4b5e669e413"
server_key_version: "encryptionkey@0"
I20260811 22:35:08.755085 24081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:35:08.757751 24118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:08.759065 24081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:35:08.759222 24081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "dc17cfbc27fa453e8c73da1023af0f3f"
format_stamp: "Formatted at 2026-08-11 22:35:08 on dist-test-slave-8hhm"
server_key: "39d19a5e37448e71831b554918579daa"
server_key_iv: "2a94ece07b0ae1b819c4d4b5e669e413"
server_key_version: "encryptionkey@0"
I20260811 22:35:08.759347 24081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:08.764279 24081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:08.767573 24081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:08.767803 24081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:08.775198 24081 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43355
I20260811 22:35:08.775234 24217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43355 every 8 connection(s)
I20260811 22:35:08.776230 24081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:08.776654 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24081
I20260811 22:35:08.776829 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:08.777148 19278 external_mini_cluster.cc:1428] Setting key 13fbb0741d6ea45ba9317f63327db780
I20260811 22:35:08.779991 24220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:08.783217 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:08.778760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57759 (local address 127.0.0.1:43355)
0811 22:35:08.779203 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:08.779223 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:08.779267 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:35:08.780156 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:08.780181 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:08.780543 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:08.780916 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:08.781776 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:08.781782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:08.781797 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:08.781810 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:08.781874 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:08.781900 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:08.782042 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:08.782127 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 22:35:08.782280 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
I20260811 22:35:08.786418 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:08.787194 24220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f: Bootstrap starting.
I20260811 22:35:08.789980 24220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:08.790424 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:08.788621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39459 (local address 127.0.0.1:43355)
0811 22:35:08.788784 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:08.788788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:08.788962 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:35:08.789098 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:08.789101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:08.789133 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:08.789204 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:08.789992 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:08.789995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:08.789997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:08.790008 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:08.790033 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:08.790044 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:08.790153 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:08.790251 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 22:35:08.790295 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:35:08.791061 24220 log.cc:824] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:08.793340 24220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f: No bootstrap required, opened a new log
I20260811 22:35:08.797384 24220 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260811 22:35:08.797645 24220 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:08.797724 24220 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc17cfbc27fa453e8c73da1023af0f3f, State: Initialized, Role: FOLLOWER
I20260811 22:35:08.798221 24220 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260811 22:35:08.798373 24220 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:08.798470 24220 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:08.798601 24220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:08.799588 24220 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260811 22:35:08.799965 24220 leader_election.cc:302] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc17cfbc27fa453e8c73da1023af0f3f; no voters:
I20260811 22:35:08.800318 24220 leader_election.cc:288] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:08.800457 24244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:08.800722 24244 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [term 1 LEADER]: Becoming Leader. State: Replica: dc17cfbc27fa453e8c73da1023af0f3f, State: Running, Role: LEADER
I20260811 22:35:08.801065 24244 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260811 22:35:08.801687 24220 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:08.802954 24246 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } } }
I20260811 22:35:08.803099 24246 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:08.803366 24249 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc17cfbc27fa453e8c73da1023af0f3f. Latest consensus state: current_term: 1 leader_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc17cfbc27fa453e8c73da1023af0f3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } } }
I20260811 22:35:08.803459 24249 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f [sys.catalog]: This master's current role is: LEADER
W20260811 22:35:08.806674 24275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:08.806773 24275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:08.806845 24265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:08.807549 24265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:08.814007 24265 catalog_manager.cc:1347] Generated new cluster ID: d22583f613514ecbbfff6a7e3b1b65df
I20260811 22:35:08.814085 24265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:08.837558 24265 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:08.837971 24265 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:08.853329 24265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc17cfbc27fa453e8c73da1023af0f3f: Generated new TSK 0
I20260811 22:35:08.854282 24265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:08.868913 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24081
2026-08-11T22:35:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1711 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:08 dist-test-slave-8hhm krb5kdc[24286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:08 dist-test-slave-8hhm krb5kdc[24286](info): set up 2 sockets
Aug 11 22:35:08 dist-test-slave-8hhm krb5kdc[24286](info): commencing operation
krb5kdc: starting...
W20260811 22:35:10.187780 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:10 dist-test-slave-8hhm krb5kdc[24286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:10.330475 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:10.464214 24311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:10.464551 24311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:10.464613 24311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:10.469089 24311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:10.469184 24311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:10.469271 24311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:10.469312 24311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:10.469353 24311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:10.475059 24311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:10.476274 24311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:10.477416 24311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:10.487670 24317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:10.487660 24325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:10.487668 24318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:10.488082 24311 server_base.cc:1034] running on GCE node
I20260811 22:35:10.488749 24311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:10.490013 24311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:10.491196 24311 hybrid_clock.cc:648] HybridClock initialized: now 1786487710491185 us; error 29 us; skew 500 ppm
Aug 11 22:35:10 dist-test-slave-8hhm krb5kdc[24286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:10.498416 24311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:10.499478 24311 webserver.cc:460] Webserver started at http://127.0.0.1:45193/ using document root <none> and password file <none>
I20260811 22:35:10.500185 24311 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:10.500254 24311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:10.500525 24311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:10.502921 24311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "5a2398bffa864433b6813f7e3fc88d5c"
format_stamp: "Formatted at 2026-08-11 22:35:10 on dist-test-slave-8hhm"
server_key: "0d2efbe4d7b6c8f4ba6123ea6bbef102"
server_key_iv: "590fb553445eecd5db9a6c95759c4d1e"
server_key_version: "encryptionkey@0"
I20260811 22:35:10.503522 24311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "5a2398bffa864433b6813f7e3fc88d5c"
format_stamp: "Formatted at 2026-08-11 22:35:10 on dist-test-slave-8hhm"
server_key: "0d2efbe4d7b6c8f4ba6123ea6bbef102"
server_key_iv: "590fb553445eecd5db9a6c95759c4d1e"
server_key_version: "encryptionkey@0"
I20260811 22:35:10.507612 24311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 22:35:10.510272 24333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:10.511538 24311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:35:10.511693 24311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "5a2398bffa864433b6813f7e3fc88d5c"
format_stamp: "Formatted at 2026-08-11 22:35:10 on dist-test-slave-8hhm"
server_key: "0d2efbe4d7b6c8f4ba6123ea6bbef102"
server_key_iv: "590fb553445eecd5db9a6c95759c4d1e"
server_key_version: "encryptionkey@0"
I20260811 22:35:10.511826 24311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:10.530566 24311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:10.533756 24311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:10.533985 24311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:10.541996 24311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34763
I20260811 22:35:10.542047 24438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34763 every 8 connection(s)
I20260811 22:35:10.543097 24311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:10.546250 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:10.548790 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24311
I20260811 22:35:10.548940 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:10.549270 19278 external_mini_cluster.cc:1428] Setting key 2704d1cefd9ce2de904b09c04194db28
I20260811 22:35:10.553462 24440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c: Bootstrap starting.
I20260811 22:35:10.556404 24440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:10.557304 24440 log.cc:824] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:35:10 dist-test-slave-8hhm krb5kdc[24286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:35:10.559799 24440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c: No bootstrap required, opened a new log
I20260811 22:35:10.563879 24440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2398bffa864433b6813f7e3fc88d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34763 } }
I20260811 22:35:10.564181 24440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:10.564246 24440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a2398bffa864433b6813f7e3fc88d5c, State: Initialized, Role: FOLLOWER
I20260811 22:35:10.564817 24440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2398bffa864433b6813f7e3fc88d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34763 } }
I20260811 22:35:10.565059 24440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:10.565147 24440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:10.565297 24440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:10.565641 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:10.550859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:37151 (local address 127.0.0.1:34763)
0811 22:35:10.551315 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:10.551336 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:10.551380 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:35:10.552304 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:10.552333 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:10.552794 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:10.553180 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:10.554037 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:10.555606 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:10.556580 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:10.557118 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:10.557375 (+ 257us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:10.560376 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:10.560382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:10.560396 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:10.560424 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:10.562281 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:10.562841 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:10.562846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:10.562851 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:10.562972 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:10.563341 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:10.563351 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:10.563352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:10.563762 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:10.564013 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:10.564306 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:35:10.564594 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260811 22:35:10.566516 24440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2398bffa864433b6813f7e3fc88d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34763 } }
I20260811 22:35:10.566947 24440 leader_election.cc:302] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a2398bffa864433b6813f7e3fc88d5c; no voters:
I20260811 22:35:10.567354 24440 leader_election.cc:288] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:10.567474 24453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:10.567737 24453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [term 1 LEADER]: Becoming Leader. State: Replica: 5a2398bffa864433b6813f7e3fc88d5c, State: Running, Role: LEADER
I20260811 22:35:10.568082 24453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2398bffa864433b6813f7e3fc88d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34763 } }
I20260811 22:35:10.568756 24440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:10.569339 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:10.569830 24453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a2398bffa864433b6813f7e3fc88d5c. Latest consensus state: current_term: 1 leader_uuid: "5a2398bffa864433b6813f7e3fc88d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2398bffa864433b6813f7e3fc88d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34763 } } }
I20260811 22:35:10.569964 24453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:10.570328 24464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:10.572050 24455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a2398bffa864433b6813f7e3fc88d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2398bffa864433b6813f7e3fc88d5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34763 } } }
I20260811 22:35:10.572165 24455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:10.573961 24464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:10.581079 24464 catalog_manager.cc:1347] Generated new cluster ID: 4aa120d90a8b44a3a0b2d11a5cd90099
I20260811 22:35:10.581158 24464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:10.582643 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:10.572013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:43101 (local address 127.0.0.1:34763)
0811 22:35:10.572139 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:10.572145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:10.573062 (+ 917us) server_negotiation.cc:407] Connection header received
0811 22:35:10.573261 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:10.573264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:10.573325 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:10.573425 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:10.574443 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:10.574999 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:10.576056 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:10.576413 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:10.576538 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:10.578618 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:10.578623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:10.578626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:10.578657 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:10.580417 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:10.581003 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:10.581006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:10.581008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:10.581111 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:10.581593 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:10.581597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:10.581599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:10.581812 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:10.581998 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:10.582307 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:35:10.582418 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:35:10.626688 24464 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:10.627718 24464 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:10.634851 24464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a2398bffa864433b6813f7e3fc88d5c: Generated new TSK 0
I20260811 22:35:10.635749 24464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:10.651523 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24311
W20260811 22:35:10.674091 24283 connection.cc:535] client connection to 127.0.0.1:34763 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34763): Connection reset by peer (error 104)
2026-08-11T22:35:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1782 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:10 dist-test-slave-8hhm krb5kdc[24493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:10 dist-test-slave-8hhm krb5kdc[24493](info): set up 2 sockets
Aug 11 22:35:10 dist-test-slave-8hhm krb5kdc[24493](info): commencing operation
krb5kdc: starting...
W20260811 22:35:11.960970 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:12 dist-test-slave-8hhm krb5kdc[24493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:12.111928 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:12.248464 24520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:12.248767 24520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:12.248811 24520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:12.252974 24520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:12.253053 24520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:12.253074 24520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:12.253095 24520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:12.253111 24520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:12.258559 24520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:12.259660 24520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:12.260695 24520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:12.269476 24532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:12.269517 24529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:12.269483 24541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:12.269585 24520 server_base.cc:1034] running on GCE node
I20260811 22:35:12.270346 24520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:12.271515 24520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:12.272775 24520 hybrid_clock.cc:648] HybridClock initialized: now 1786487712272759 us; error 27 us; skew 500 ppm
Aug 11 22:35:12 dist-test-slave-8hhm krb5kdc[24493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:12.275921 24520 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:12.276830 24520 webserver.cc:460] Webserver started at http://127.0.0.1:40521/ using document root <none> and password file <none>
I20260811 22:35:12.277463 24520 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:12.277532 24520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:12.277719 24520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:12.279819 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "5ff35549d25b411aa773051025824764"
format_stamp: "Formatted at 2026-08-11 22:35:12 on dist-test-slave-8hhm"
server_key: "41f85d5719e30ff62e9a42d8661454b6"
server_key_iv: "0dd392f914b33f8c94c4e2b605f36644"
server_key_version: "encryptionkey@0"
I20260811 22:35:12.280364 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "5ff35549d25b411aa773051025824764"
format_stamp: "Formatted at 2026-08-11 22:35:12 on dist-test-slave-8hhm"
server_key: "41f85d5719e30ff62e9a42d8661454b6"
server_key_iv: "0dd392f914b33f8c94c4e2b605f36644"
server_key_version: "encryptionkey@0"
I20260811 22:35:12.284365 24520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:35:12.286868 24561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:12.288132 24520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:35:12.288268 24520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "5ff35549d25b411aa773051025824764"
format_stamp: "Formatted at 2026-08-11 22:35:12 on dist-test-slave-8hhm"
server_key: "41f85d5719e30ff62e9a42d8661454b6"
server_key_iv: "0dd392f914b33f8c94c4e2b605f36644"
server_key_version: "encryptionkey@0"
I20260811 22:35:12.288376 24520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:12.305935 24520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:12.308904 24520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:12.309134 24520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:12.316642 24520 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42701
I20260811 22:35:12.316668 24665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42701 every 8 connection(s)
I20260811 22:35:12.317680 24520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:12.320564 24666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:12.320752 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24520
I20260811 22:35:12.320909 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:12.321259 19278 external_mini_cluster.cc:1428] Setting key 6bd2777d33c925dc04b068f24c3e7e9c
I20260811 22:35:12.328137 24666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764: Bootstrap starting.
I20260811 22:35:12.330963 24666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:12.331772 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:12.322753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:49033 (local address 127.0.0.1:42701)
0811 22:35:12.323181 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:12.323200 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:12.323244 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:35:12.324124 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:12.324152 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:12.324623 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:12.325017 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:12.325464 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:12.327016 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:12.327922 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:12.328603 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:12.328778 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:12.329713 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:12.329719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:12.329736 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:12.330235 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:12.330304 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:12.330335 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:12.330472 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:12.330645 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:35:12.330790 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":109,"threads_started":1}
I20260811 22:35:12.331902 24666 log.cc:824] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:12.334182 24666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764: No bootstrap required, opened a new log
I20260811 22:35:12.334936 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:12.338209 24666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff35549d25b411aa773051025824764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260811 22:35:12.338438 24666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:12.338501 24666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff35549d25b411aa773051025824764, State: Initialized, Role: FOLLOWER
I20260811 22:35:12.339030 24666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff35549d25b411aa773051025824764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260811 22:35:12.339213 24666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:12.339283 24666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:12.339381 24666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:12.340590 24666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff35549d25b411aa773051025824764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260811 22:35:12.341032 24666 leader_election.cc:302] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ff35549d25b411aa773051025824764; no voters:
I20260811 22:35:12.341506 24666 leader_election.cc:288] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:12.341912 24690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:12.341943 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:12.337074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:35775 (local address 127.0.0.1:42701)
0811 22:35:12.337203 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:12.337208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:12.337405 (+ 197us) server_negotiation.cc:407] Connection header received
0811 22:35:12.337563 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:12.337567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:12.337617 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:12.337699 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:12.337974 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:12.338680 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:12.339859 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:12.340129 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:12.340214 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:12.341323 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:12.341327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:12.341330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:12.341346 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:12.341378 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:12.341390 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:12.341471 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:12.341686 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 22:35:12.341772 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:35:12.342116 24690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff35549d25b411aa773051025824764, State: Running, Role: LEADER
I20260811 22:35:12.342478 24690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff35549d25b411aa773051025824764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260811 22:35:12.342995 24666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:12.344357 24694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ff35549d25b411aa773051025824764. Latest consensus state: current_term: 1 leader_uuid: "5ff35549d25b411aa773051025824764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff35549d25b411aa773051025824764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } } }
I20260811 22:35:12.344640 24695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ff35549d25b411aa773051025824764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ff35549d25b411aa773051025824764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } } }
I20260811 22:35:12.344764 24694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:12.344848 24695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:12.345302 24716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:12.348212 24716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:12.353457 24716 catalog_manager.cc:1347] Generated new cluster ID: 5e3960128e604105b264a5718800b2b9
I20260811 22:35:12.353529 24716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:12.358489 24716 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:12.359766 24716 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:12.371879 24716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ff35549d25b411aa773051025824764: Generated new TSK 0
I20260811 22:35:12.372717 24716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:12.383519 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24520
W20260811 22:35:12.397127 24487 connection.cc:535] client connection to 127.0.0.1:42701 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42701): Connection reset by peer (error 104)
2026-08-11T22:35:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:12 dist-test-slave-8hhm krb5kdc[24736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:12 dist-test-slave-8hhm krb5kdc[24736](info): set up 2 sockets
Aug 11 22:35:12 dist-test-slave-8hhm krb5kdc[24736](info): commencing operation
krb5kdc: starting...
W20260811 22:35:13.695910 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:13 dist-test-slave-8hhm krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:13.828205 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37253
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:13.942431 24764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:13.942749 24764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:13.942842 24764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:13.946884 24764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:13.946983 24764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:13.947024 24764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:13.947084 24764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:13.947125 24764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:13.952330 24764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:13.953428 24764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:13.954492 24764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:13.963210 24775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:13.963202 24772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:13.963212 24780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:13.963488 24764 server_base.cc:1034] running on GCE node
I20260811 22:35:13.964133 24764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:13.965350 24764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:13.966543 24764 hybrid_clock.cc:648] HybridClock initialized: now 1786487713966525 us; error 30 us; skew 500 ppm
Aug 11 22:35:13 dist-test-slave-8hhm krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:13.969530 24764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:13.970539 24764 webserver.cc:460] Webserver started at http://127.0.0.1:45279/ using document root <none> and password file <none>
I20260811 22:35:13.971140 24764 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:13.971239 24764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:13.971490 24764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:13.973670 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f"
format_stamp: "Formatted at 2026-08-11 22:35:13 on dist-test-slave-8hhm"
server_key: "2b56bcef7d97e7fd373363827cddabdc"
server_key_iv: "6dac03a4a986e61f7f94440878bf36c9"
server_key_version: "encryptionkey@0"
I20260811 22:35:13.974246 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f"
format_stamp: "Formatted at 2026-08-11 22:35:13 on dist-test-slave-8hhm"
server_key: "2b56bcef7d97e7fd373363827cddabdc"
server_key_iv: "6dac03a4a986e61f7f94440878bf36c9"
server_key_version: "encryptionkey@0"
I20260811 22:35:13.978245 24764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:35:13.980707 24797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:13.981989 24764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:35:13.982156 24764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f"
format_stamp: "Formatted at 2026-08-11 22:35:13 on dist-test-slave-8hhm"
server_key: "2b56bcef7d97e7fd373363827cddabdc"
server_key_iv: "6dac03a4a986e61f7f94440878bf36c9"
server_key_version: "encryptionkey@0"
I20260811 22:35:13.982277 24764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:13.986920 24764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:13.990247 24764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:13.990456 24764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:13.997954 24764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37253
I20260811 22:35:13.997979 24912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37253 every 8 connection(s)
I20260811 22:35:13.998937 24764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:14.001808 24913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:14.005550 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 24764
I20260811 22:35:14.005699 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:14.006057 19278 external_mini_cluster.cc:1428] Setting key 017c96c557bdcdd71d1949a856f781f6
I20260811 22:35:14.007943 24913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f: Bootstrap starting.
I20260811 22:35:14.010385 24913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:14.011240 24913 log.cc:824] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:14.011777 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:14.007554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:55879 (local address 127.0.0.1:37253)
0811 22:35:14.008063 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:14.008084 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:14.008130 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:35:14.008816 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:14.008833 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:14.009117 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:14.009532 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:14.010411 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:14.010416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:14.010428 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:14.010439 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:14.010517 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:14.010548 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:14.010681 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:14.010848 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 22:35:14.011020 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":129,"threads_started":1}
I20260811 22:35:14.014242 24913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f: No bootstrap required, opened a new log
I20260811 22:35:14.014585 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:14.018133 24913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260811 22:35:14.018391 24913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:14.018497 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:14.016682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:47991 (local address 127.0.0.1:37253)
0811 22:35:14.016813 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:14.016817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:14.017021 (+ 204us) server_negotiation.cc:407] Connection header received
0811 22:35:14.017157 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:14.017159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:14.017192 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:14.017308 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:14.018037 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:14.018040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:14.018042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:14.018054 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:14.018081 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:14.018100 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:14.018192 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:14.018286 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 22:35:14.018328 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:35:14.018509 24913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1df6ca6cf85d4e7ebea661e3da84bd6f, State: Initialized, Role: FOLLOWER
I20260811 22:35:14.018990 24913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260811 22:35:14.019150 24913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:14.019238 24913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:14.019358 24913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:14.020262 24913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260811 22:35:14.020617 24913 leader_election.cc:302] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1df6ca6cf85d4e7ebea661e3da84bd6f; no voters:
I20260811 22:35:14.020955 24913 leader_election.cc:288] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:14.021067 24933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:14.021354 24933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [term 1 LEADER]: Becoming Leader. State: Replica: 1df6ca6cf85d4e7ebea661e3da84bd6f, State: Running, Role: LEADER
I20260811 22:35:14.021755 24933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260811 22:35:14.022257 24913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:14.023659 24934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } } }
I20260811 22:35:14.023677 24935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1df6ca6cf85d4e7ebea661e3da84bd6f. Latest consensus state: current_term: 1 leader_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1df6ca6cf85d4e7ebea661e3da84bd6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } } }
I20260811 22:35:14.023813 24935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:14.023813 24934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f [sys.catalog]: This master's current role is: LEADER
W20260811 22:35:14.027082 24960 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:14.027155 24960 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:14.027245 24948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:14.028028 24948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:14.034384 24948 catalog_manager.cc:1347] Generated new cluster ID: 4fd654f8d86043479a36d8f57961f109
I20260811 22:35:14.034523 24948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:14.050043 24948 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:14.050457 24948 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:14.055732 24948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1df6ca6cf85d4e7ebea661e3da84bd6f: Generated new TSK 0
I20260811 22:35:14.056425 24948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:14.062781 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 24764
2026-08-11T22:35:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1683 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:14 dist-test-slave-8hhm krb5kdc[24978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:14 dist-test-slave-8hhm krb5kdc[24978](info): set up 2 sockets
Aug 11 22:35:14 dist-test-slave-8hhm krb5kdc[24978](info): commencing operation
krb5kdc: starting...
W20260811 22:35:15.375861 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:15 dist-test-slave-8hhm krb5kdc[24978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:15.532384 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38955
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:15.649736 25006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:15.650089 25006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:15.650177 25006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:15.654309 25006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:15.654404 25006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:15.654440 25006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:15.654486 25006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:15.654522 25006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:15.660244 25006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:15.661521 25006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:15.662580 25006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:15.671368 25016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:15.671377 25014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:15.671588 25006 server_base.cc:1034] running on GCE node
W20260811 22:35:15.671378 25020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:15.672250 25006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:15.673473 25006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:15.674636 25006 hybrid_clock.cc:648] HybridClock initialized: now 1786487715674603 us; error 45 us; skew 500 ppm
Aug 11 22:35:15 dist-test-slave-8hhm krb5kdc[24978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:15.682363 25006 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:15.683282 25006 webserver.cc:460] Webserver started at http://127.0.0.1:36631/ using document root <none> and password file <none>
I20260811 22:35:15.683827 25006 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:15.683888 25006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:15.684060 25006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:15.686240 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4"
format_stamp: "Formatted at 2026-08-11 22:35:15 on dist-test-slave-8hhm"
server_key: "48ae05d388a4003c153bf0fb7fdd5c0d"
server_key_iv: "6c79126a5eec13ad217d00f986736696"
server_key_version: "encryptionkey@0"
I20260811 22:35:15.686795 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4"
format_stamp: "Formatted at 2026-08-11 22:35:15 on dist-test-slave-8hhm"
server_key: "48ae05d388a4003c153bf0fb7fdd5c0d"
server_key_iv: "6c79126a5eec13ad217d00f986736696"
server_key_version: "encryptionkey@0"
I20260811 22:35:15.690608 25006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:35:15.692962 25035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:15.694342 25006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:35:15.694489 25006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4"
format_stamp: "Formatted at 2026-08-11 22:35:15 on dist-test-slave-8hhm"
server_key: "48ae05d388a4003c153bf0fb7fdd5c0d"
server_key_iv: "6c79126a5eec13ad217d00f986736696"
server_key_version: "encryptionkey@0"
I20260811 22:35:15.694585 25006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:15.715122 25006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:15.718308 25006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:15.718525 25006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:15.726356 25006 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38955
I20260811 22:35:15.726367 25148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38955 every 8 connection(s)
I20260811 22:35:15.727434 25006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:15.730172 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25006
I20260811 22:35:15.730351 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:15.730545 25149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:15.730731 19278 external_mini_cluster.cc:1428] Setting key 62842ff9a28e2a163f11dad155f77627
I20260811 22:35:15.737954 25149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4: Bootstrap starting.
Aug 11 22:35:15 dist-test-slave-8hhm krb5kdc[24978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:35:15.740530 25149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:15.741394 25149 log.cc:824] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:15.743494 25149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4: No bootstrap required, opened a new log
I20260811 22:35:15.746345 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:15.732266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:42547 (local address 127.0.0.1:38955)
0811 22:35:15.732717 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:15.732734 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:15.732775 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:35:15.733541 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:15.733562 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:15.733920 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:15.734222 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:15.735223 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:15.736251 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:15.737492 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:15.738001 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:15.738227 (+ 226us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:15.741329 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:15.741334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:15.741348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:15.741378 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:15.743319 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:15.743933 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:15.743938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:15.743943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:15.744058 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:15.744420 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:15.744429 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:15.744430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:15.744820 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:15.745061 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:15.745409 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:35:15.745627 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":156,"threads_started":1}
I20260811 22:35:15.747360 25149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38955 } }
I20260811 22:35:15.747633 25149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:15.747737 25149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa70c5e0060443e9c1cd69d78e1e4f4, State: Initialized, Role: FOLLOWER
I20260811 22:35:15.748190 25149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38955 } }
I20260811 22:35:15.748373 25149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:15.748508 25149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:15.748656 25149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:15.749576 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:15.749881 25149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38955 } }
I20260811 22:35:15.750234 25149 leader_election.cc:302] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fa70c5e0060443e9c1cd69d78e1e4f4; no voters:
I20260811 22:35:15.750569 25149 leader_election.cc:288] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:15.750695 25157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:15.750976 25157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [term 1 LEADER]: Becoming Leader. State: Replica: 3fa70c5e0060443e9c1cd69d78e1e4f4, State: Running, Role: LEADER
I20260811 22:35:15.751355 25157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38955 } }
I20260811 22:35:15.751991 25149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:15.753122 25162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38955 } } }
I20260811 22:35:15.753543 25162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:15.753433 25164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fa70c5e0060443e9c1cd69d78e1e4f4. Latest consensus state: current_term: 1 leader_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa70c5e0060443e9c1cd69d78e1e4f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38955 } } }
I20260811 22:35:15.753904 25164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4 [sys.catalog]: This master's current role is: LEADER
W20260811 22:35:15.758085 25210 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:15.758185 25210 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:15.758307 25184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:15.759228 25184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:15.762115 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:15.751851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:52623 (local address 127.0.0.1:38955)
0811 22:35:15.751973 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:15.751978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:15.752405 (+ 427us) server_negotiation.cc:407] Connection header received
0811 22:35:15.752528 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:15.752531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:15.752585 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:15.752676 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:15.753806 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:15.754602 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:15.755761 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:15.755986 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:15.756068 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:15.757913 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:15.757918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:15.757920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:15.757947 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:15.759795 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:15.760407 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:15.760410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:15.760412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:15.760504 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:15.760939 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:15.760943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:15.760945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:15.761168 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:15.761359 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:15.761836 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 22:35:15.761948 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:35:15.765487 25184 catalog_manager.cc:1347] Generated new cluster ID: d6cc2fddde184f16b29fb44e4f2c282a
I20260811 22:35:15.765556 25184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:15.774993 25184 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:15.776185 25184 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:15.791771 25184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fa70c5e0060443e9c1cd69d78e1e4f4: Generated new TSK 0
I20260811 22:35:15.792557 25184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:15.802299 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25006
W20260811 22:35:15.823768 24973 connection.cc:535] client connection to 127.0.0.1:38955 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38955): Connection reset by peer (error 104)
2026-08-11T22:35:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1742 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:15 dist-test-slave-8hhm krb5kdc[25217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:15 dist-test-slave-8hhm krb5kdc[25217](info): set up 2 sockets
Aug 11 22:35:15 dist-test-slave-8hhm krb5kdc[25217](info): commencing operation
krb5kdc: starting...
W20260811 22:35:17.120230 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:17 dist-test-slave-8hhm krb5kdc[25217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:17.256915 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:17.376188 25238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:17.376493 25238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:17.376535 25238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:17.380580 25238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:17.380654 25238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:17.380673 25238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:17.380690 25238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:17.380705 25238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:17.386096 25238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:17.387190 25238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:17.388236 25238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:17.397099 25248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:17.397086 25247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:17.397069 25253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:17.397339 25238 server_base.cc:1034] running on GCE node
I20260811 22:35:17.398088 25238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:17.399240 25238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:17.400451 25238 hybrid_clock.cc:648] HybridClock initialized: now 1786487717400433 us; error 37 us; skew 500 ppm
Aug 11 22:35:17 dist-test-slave-8hhm krb5kdc[25217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:17.403348 25238 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:17.404325 25238 webserver.cc:460] Webserver started at http://127.0.0.1:42443/ using document root <none> and password file <none>
I20260811 22:35:17.404928 25238 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:17.405026 25238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:17.405347 25238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:17.407570 25238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "959bf80cdc804721a7dbcf91cd0869e8"
format_stamp: "Formatted at 2026-08-11 22:35:17 on dist-test-slave-8hhm"
server_key: "f3aa95a0ec32708a8762d06c1e47d5ae"
server_key_iv: "6b44ccfc0442e57c5e4fd9c8db1b30f4"
server_key_version: "encryptionkey@0"
I20260811 22:35:17.408167 25238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "959bf80cdc804721a7dbcf91cd0869e8"
format_stamp: "Formatted at 2026-08-11 22:35:17 on dist-test-slave-8hhm"
server_key: "f3aa95a0ec32708a8762d06c1e47d5ae"
server_key_iv: "6b44ccfc0442e57c5e4fd9c8db1b30f4"
server_key_version: "encryptionkey@0"
I20260811 22:35:17.412043 25238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:35:17.414649 25267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:17.416002 25238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:35:17.416169 25238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "959bf80cdc804721a7dbcf91cd0869e8"
format_stamp: "Formatted at 2026-08-11 22:35:17 on dist-test-slave-8hhm"
server_key: "f3aa95a0ec32708a8762d06c1e47d5ae"
server_key_iv: "6b44ccfc0442e57c5e4fd9c8db1b30f4"
server_key_version: "encryptionkey@0"
I20260811 22:35:17.416299 25238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:17.430039 25238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:17.433429 25238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:17.433647 25238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:17.441242 25238 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38429
I20260811 22:35:17.441303 25367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38429 every 8 connection(s)
I20260811 22:35:17.442368 25238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:17.445036 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25238
I20260811 22:35:17.445185 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:17.445458 25370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:17.445524 19278 external_mini_cluster.cc:1428] Setting key d980bf8ac6185aa0ad48fa46346dff84
I20260811 22:35:17.452370 25370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8: Bootstrap starting.
I20260811 22:35:17.455001 25370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:17.455089 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:17.447032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:48635 (local address 127.0.0.1:38429)
0811 22:35:17.447509 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:17.447524 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:17.447560 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:35:17.448304 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:17.448324 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:17.448685 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:17.449068 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:17.449532 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:17.450537 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:17.451569 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:17.452102 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:17.452290 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:17.453325 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:17.453330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:17.453343 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:17.453753 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:17.453811 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:17.453834 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:17.453981 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:17.454187 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:35:17.454330 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":158,"threads_started":1}
I20260811 22:35:17.455858 25370 log.cc:824] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:17.458093 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:17.458112 25370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8: No bootstrap required, opened a new log
I20260811 22:35:17.462700 25370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959bf80cdc804721a7dbcf91cd0869e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38429 } }
I20260811 22:35:17.463063 25370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:17.463195 25370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959bf80cdc804721a7dbcf91cd0869e8, State: Initialized, Role: FOLLOWER
I20260811 22:35:17.463769 25370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959bf80cdc804721a7dbcf91cd0869e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38429 } }
I20260811 22:35:17.463989 25370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:17.464089 25370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:17.464211 25370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:17.464717 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:17.460152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:33531 (local address 127.0.0.1:38429)
0811 22:35:17.460275 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:17.460279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:17.460574 (+ 295us) server_negotiation.cc:407] Connection header received
0811 22:35:17.460722 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:17.460724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:17.460763 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:17.460830 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:17.461160 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:17.461707 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:17.462824 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:17.463041 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:17.463159 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:17.464192 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:17.464196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:17.464199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:17.464214 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:17.464248 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:17.464260 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:17.464375 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:17.464527 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 22:35:17.464563 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:35:17.465533 25370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959bf80cdc804721a7dbcf91cd0869e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38429 } }
I20260811 22:35:17.465914 25370 leader_election.cc:302] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 959bf80cdc804721a7dbcf91cd0869e8; no voters:
I20260811 22:35:17.466193 25370 leader_election.cc:288] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:17.466321 25401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:17.466563 25401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [term 1 LEADER]: Becoming Leader. State: Replica: 959bf80cdc804721a7dbcf91cd0869e8, State: Running, Role: LEADER
I20260811 22:35:17.466930 25401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959bf80cdc804721a7dbcf91cd0869e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38429 } }
I20260811 22:35:17.467500 25370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:17.468909 25402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "959bf80cdc804721a7dbcf91cd0869e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959bf80cdc804721a7dbcf91cd0869e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38429 } } }
I20260811 22:35:17.469046 25402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:17.469282 25403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959bf80cdc804721a7dbcf91cd0869e8. Latest consensus state: current_term: 1 leader_uuid: "959bf80cdc804721a7dbcf91cd0869e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959bf80cdc804721a7dbcf91cd0869e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38429 } } }
I20260811 22:35:17.469650 25403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8 [sys.catalog]: This master's current role is: LEADER
W20260811 22:35:17.472262 25427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:17.472359 25427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:17.472453 25423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:17.473264 25423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:17.478521 25423 catalog_manager.cc:1347] Generated new cluster ID: d436e35f977d424885abb04f48a13d9b
I20260811 22:35:17.478590 25423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:17.497001 25423 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:17.498117 25423 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:17.508777 25423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 959bf80cdc804721a7dbcf91cd0869e8: Generated new TSK 0
I20260811 22:35:17.509634 25423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:17.525475 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25238
W20260811 22:35:17.544456 25215 connection.cc:535] client connection to 127.0.0.1:38429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38429): Connection reset by peer (error 104)
2026-08-11T22:35:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1720 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:17 dist-test-slave-8hhm krb5kdc[25438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:17 dist-test-slave-8hhm krb5kdc[25438](info): set up 2 sockets
Aug 11 22:35:17 dist-test-slave-8hhm krb5kdc[25438](info): commencing operation
krb5kdc: starting...
W20260811 22:35:18.842499 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:18 dist-test-slave-8hhm krb5kdc[25438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:35:18.984023 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:19.103807 25461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:19.104143 25461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:19.104231 25461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:19.108403 25461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:19.108513 25461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:19.108551 25461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:19.108592 25461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:19.108623 25461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:19.114279 25461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:19.115491 25461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:19.116648 25461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:35:19.126000 25461 server_base.cc:1034] running on GCE node
W20260811 22:35:19.125972 25474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:19.126036 25468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:19.126183 25476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:19.126819 25461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:19.128048 25461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:19.129302 25461 hybrid_clock.cc:648] HybridClock initialized: now 1786487719129279 us; error 32 us; skew 500 ppm
Aug 11 22:35:19 dist-test-slave-8hhm krb5kdc[25438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487719, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:19.132262 25461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:35:19.133399 25461 webserver.cc:460] Webserver started at http://127.0.0.1:44161/ using document root <none> and password file <none>
I20260811 22:35:19.134008 25461 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:19.134106 25461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:19.134361 25461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:19.136605 25461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "cb295ab7c26a453da6d28eaa594ce285"
format_stamp: "Formatted at 2026-08-11 22:35:19 on dist-test-slave-8hhm"
server_key: "037a18f306aa805e41ef5ccbf7d7b53e"
server_key_iv: "1d3b5c3f1c0a081bfd11d8ef85ecef16"
server_key_version: "encryptionkey@0"
I20260811 22:35:19.137318 25461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "cb295ab7c26a453da6d28eaa594ce285"
format_stamp: "Formatted at 2026-08-11 22:35:19 on dist-test-slave-8hhm"
server_key: "037a18f306aa805e41ef5ccbf7d7b53e"
server_key_iv: "1d3b5c3f1c0a081bfd11d8ef85ecef16"
server_key_version: "encryptionkey@0"
I20260811 22:35:19.141549 25461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:35:19.144248 25494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:19.145648 25461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:35:19.145823 25461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "cb295ab7c26a453da6d28eaa594ce285"
format_stamp: "Formatted at 2026-08-11 22:35:19 on dist-test-slave-8hhm"
server_key: "037a18f306aa805e41ef5ccbf7d7b53e"
server_key_iv: "1d3b5c3f1c0a081bfd11d8ef85ecef16"
server_key_version: "encryptionkey@0"
I20260811 22:35:19.145959 25461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:19.151057 25461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:19.154829 25461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:19.155061 25461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:19.162959 25461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43861
I20260811 22:35:19.162985 25605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43861 every 8 connection(s)
I20260811 22:35:19.163987 25461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:19.166932 25608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:19.172061 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25461
I20260811 22:35:19.172228 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:19.172554 19278 external_mini_cluster.cc:1428] Setting key 295032d92c80aa746bc576e1ddfd9f14
I20260811 22:35:19.173254 25608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285: Bootstrap starting.
I20260811 22:35:19.175834 25608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:19.176788 25608 log.cc:824] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:19.178128 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:19.174078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:55017 (local address 127.0.0.1:43861)
0811 22:35:19.174566 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:19.174581 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:19.174619 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:35:19.175285 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:19.175303 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:19.175601 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:19.175963 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:19.176833 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:19.176838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:19.176851 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:19.176862 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:19.176939 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:19.176969 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:19.177099 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:19.177232 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 22:35:19.177412 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":141,"threads_started":1}
I20260811 22:35:19.178977 25608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285: No bootstrap required, opened a new log
I20260811 22:35:19.181007 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:19.183554 25608 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb295ab7c26a453da6d28eaa594ce285" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 22:35:19.183921 25608 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:19.184003 25608 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb295ab7c26a453da6d28eaa594ce285, State: Initialized, Role: FOLLOWER
I20260811 22:35:19.184530 25608 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb295ab7c26a453da6d28eaa594ce285" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 22:35:19.184716 25608 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:19.184788 25608 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:19.184904 25608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:19.185549 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:19.183571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:42483 (local address 127.0.0.1:43861)
0811 22:35:19.183712 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:19.183719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:19.183864 (+ 145us) server_negotiation.cc:407] Connection header received
0811 22:35:19.184039 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:19.184044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:19.184089 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:19.184154 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:19.185091 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:19.185094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:19.185096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:19.185107 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:19.185134 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:19.185144 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:19.185261 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:19.185366 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 22:35:19.185404 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:35:19.186184 25608 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb295ab7c26a453da6d28eaa594ce285" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 22:35:19.186568 25608 leader_election.cc:302] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb295ab7c26a453da6d28eaa594ce285; no voters:
I20260811 22:35:19.186921 25608 leader_election.cc:288] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:19.187067 25626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:19.187322 25626 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [term 1 LEADER]: Becoming Leader. State: Replica: cb295ab7c26a453da6d28eaa594ce285, State: Running, Role: LEADER
I20260811 22:35:19.187691 25626 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb295ab7c26a453da6d28eaa594ce285" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 22:35:19.188274 25608 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:19.189594 25627 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb295ab7c26a453da6d28eaa594ce285" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb295ab7c26a453da6d28eaa594ce285" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } } }
I20260811 22:35:19.189735 25627 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:19.190079 25628 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb295ab7c26a453da6d28eaa594ce285. Latest consensus state: current_term: 1 leader_uuid: "cb295ab7c26a453da6d28eaa594ce285" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb295ab7c26a453da6d28eaa594ce285" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } } }
I20260811 22:35:19.190158 25648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:19.190174 25628 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:19.194048 25648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:19.200066 25648 catalog_manager.cc:1347] Generated new cluster ID: eea7540fdd9a43c193631a64bea19308
I20260811 22:35:19.200151 25648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:19.213902 25648 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:19.214262 25648 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:19.220587 25648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb295ab7c26a453da6d28eaa594ce285: Generated new TSK 0
I20260811 22:35:19.221567 25648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:19.232647 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25461
2026-08-11T22:35:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1704 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:19 dist-test-slave-8hhm krb5kdc[25671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:19 dist-test-slave-8hhm krb5kdc[25671](info): set up 2 sockets
Aug 11 22:35:19 dist-test-slave-8hhm krb5kdc[25671](info): commencing operation
krb5kdc: starting...
W20260811 22:35:20.538623 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:20 dist-test-slave-8hhm krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:20.672770 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34601
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:35967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:20.788360 25697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:20.788743 25697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:20.788834 25697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:20.793007 25697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:20.793118 25697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:20.793159 25697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:20.793218 25697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:20.793277 25697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:20.798831 25697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:35967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:34601
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:34601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:20.800014 25697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:20.801142 25697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:20.809919 25707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:20.809912 25704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:20.810204 25705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:20.809979 25697 server_base.cc:1034] running on GCE node
I20260811 22:35:20.810844 25697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:20.812023 25697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:20.813274 25697 hybrid_clock.cc:648] HybridClock initialized: now 1786487720813254 us; error 29 us; skew 500 ppm
Aug 11 22:35:20 dist-test-slave-8hhm krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:20.822126 25697 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:20.823256 25697 webserver.cc:460] Webserver started at http://127.18.211.190:34443/ using document root <none> and password file <none>
I20260811 22:35:20.823910 25697 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:20.824016 25697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:20.824290 25697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:20.826666 25697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "0e985710493f496ba95bb1e367f50784"
format_stamp: "Formatted at 2026-08-11 22:35:20 on dist-test-slave-8hhm"
server_key: "822d14efe4c4b1a1664a2d14f533533c"
server_key_iv: "5b7aa47dffe51ece469a8bd666f2d6f2"
server_key_version: "encryptionkey@0"
I20260811 22:35:20.827323 25697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "0e985710493f496ba95bb1e367f50784"
format_stamp: "Formatted at 2026-08-11 22:35:20 on dist-test-slave-8hhm"
server_key: "822d14efe4c4b1a1664a2d14f533533c"
server_key_iv: "5b7aa47dffe51ece469a8bd666f2d6f2"
server_key_version: "encryptionkey@0"
I20260811 22:35:20.831513 25697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:35:20.834272 25715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:20.835588 25697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:35:20.835764 25697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "0e985710493f496ba95bb1e367f50784"
format_stamp: "Formatted at 2026-08-11 22:35:20 on dist-test-slave-8hhm"
server_key: "822d14efe4c4b1a1664a2d14f533533c"
server_key_iv: "5b7aa47dffe51ece469a8bd666f2d6f2"
server_key_version: "encryptionkey@0"
I20260811 22:35:20.835903 25697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:20.859987 25697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:20.863564 25697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:20.863793 25697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:20.871665 25697 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:34601
I20260811 22:35:20.871672 25821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:34601 every 8 connection(s)
I20260811 22:35:20.872825 25697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:20.875833 25822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:20.881382 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25697
I20260811 22:35:20.881572 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:20.881855 25822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784: Bootstrap starting.
I20260811 22:35:20.881934 19278 external_mini_cluster.cc:1428] Setting key a8073ec5ceee9b8b4c60073edf197916
I20260811 22:35:20.884745 25822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:20.885838 25822 log.cc:824] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:20.888247 25822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784: No bootstrap required, opened a new log
I20260811 22:35:20.893476 25822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e985710493f496ba95bb1e367f50784" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34601 } }
I20260811 22:35:20.893850 25822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:20.893931 25822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e985710493f496ba95bb1e367f50784, State: Initialized, Role: FOLLOWER
I20260811 22:35:20.894389 25822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e985710493f496ba95bb1e367f50784" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34601 } }
I20260811 22:35:20.894546 25822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:20.894622 25822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:20.894734 25822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 22:35:20 dist-test-slave-8hhm krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:20.895790 25822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e985710493f496ba95bb1e367f50784" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34601 } }
I20260811 22:35:20.896169 25822 leader_election.cc:302] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e985710493f496ba95bb1e367f50784; no voters:
I20260811 22:35:20.896530 25822 leader_election.cc:288] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:20.896658 25830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:20.896893 25830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [term 1 LEADER]: Becoming Leader. State: Replica: 0e985710493f496ba95bb1e367f50784, State: Running, Role: LEADER
I20260811 22:35:20.897282 25830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e985710493f496ba95bb1e367f50784" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34601 } }
I20260811 22:35:20.898010 25822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:20.899180 25835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e985710493f496ba95bb1e367f50784. Latest consensus state: current_term: 1 leader_uuid: "0e985710493f496ba95bb1e367f50784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e985710493f496ba95bb1e367f50784" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34601 } } }
I20260811 22:35:20.899325 25835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:20.899670 25831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e985710493f496ba95bb1e367f50784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e985710493f496ba95bb1e367f50784" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 34601 } } }
I20260811 22:35:20.899786 25831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:20.900115 25847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:20.902453 25825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:20.883640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45343 (local address 127.18.211.190:34601)
0811 22:35:20.885503 (+ 1863us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:20.885527 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:20.885573 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:35:20.886587 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:20.886616 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:20.887083 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:20.887484 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:20.889488 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:20.891239 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:20.892643 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:20.893177 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:20.893391 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:20.896325 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:20.896332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:20.896353 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:20.896391 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:20.898754 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:20.899504 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:20.899513 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:20.899522 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:20.899712 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:20.900179 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:20.900183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:20.900185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:20.900731 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:20.900959 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:20.901486 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 22:35:20.901681 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1667,"thread_start_us":135,"threads_started":1}
I20260811 22:35:20.904131 25847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:20.905620 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:20.910604 25847 catalog_manager.cc:1347] Generated new cluster ID: d4ceb56557cc4250b05da86a314dda85
I20260811 22:35:20.910674 25847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:20.918800 25825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:20.907855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55301 (local address 127.18.211.190:34601)
0811 22:35:20.907957 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:20.907961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:20.909396 (+ 1435us) server_negotiation.cc:407] Connection header received
0811 22:35:20.909530 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:20.909533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:20.909582 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:20.909671 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:20.910602 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:20.911417 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:20.912689 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:20.913196 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:20.913346 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:20.915134 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:20.915137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:20.915140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:20.915166 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:20.916868 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:20.917436 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:20.917440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:20.917442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:20.917529 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:20.917881 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:20.917884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:20.917886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:20.918097 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:20.918248 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:20.918535 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:35:20.918638 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:35:20.926555 25847 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:20.927896 25847 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:20.942619 25847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e985710493f496ba95bb1e367f50784: Generated new TSK 0
I20260811 22:35:20.943616 25847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:20.954587 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25697
W20260811 22:35:20.969235 25667 connection.cc:535] client connection to 127.18.211.190:34601 recv error: Network error: failed to read from TLS socket (remote: 127.18.211.190:34601): Connection reset by peer (error 104)
2026-08-11T22:35:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1720 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:21 dist-test-slave-8hhm krb5kdc[25888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:21 dist-test-slave-8hhm krb5kdc[25888](info): set up 2 sockets
Aug 11 22:35:21 dist-test-slave-8hhm krb5kdc[25888](info): commencing operation
krb5kdc: starting...
W20260811 22:35:22.259904 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:22 dist-test-slave-8hhm krb5kdc[25888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:22.398392 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:36793
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:39979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:36793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:22.519459 25908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:22.519769 25908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:22.519814 25908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:22.523833 25908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:22.523901 25908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:22.523922 25908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:22.523939 25908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:22.523955 25908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:22.529523 25908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:39979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:36793
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:36793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:22.530616 25908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:22.531610 25908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:35:22.540823 25908 server_base.cc:1034] running on GCE node
W20260811 22:35:22.540788 25924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:22.540781 25921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:22.540781 25922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:22.541712 25908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:22.542909 25908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:22.544098 25908 hybrid_clock.cc:648] HybridClock initialized: now 1786487722544060 us; error 57 us; skew 500 ppm
Aug 11 22:35:22 dist-test-slave-8hhm krb5kdc[25888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487722, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:22.546945 25908 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:22.547906 25908 webserver.cc:460] Webserver started at http://127.18.211.190:45167/ using document root <none> and password file <none>
I20260811 22:35:22.548498 25908 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:22.548588 25908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:22.548842 25908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:22.551010 25908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "6cb70d7beec04637a2a95edc0a396ab6"
format_stamp: "Formatted at 2026-08-11 22:35:22 on dist-test-slave-8hhm"
server_key: "b254807c6f2a11347ad1bdc34597006d"
server_key_iv: "e5d5deb91456e56c1ce0733b2be73071"
server_key_version: "encryptionkey@0"
I20260811 22:35:22.551632 25908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "6cb70d7beec04637a2a95edc0a396ab6"
format_stamp: "Formatted at 2026-08-11 22:35:22 on dist-test-slave-8hhm"
server_key: "b254807c6f2a11347ad1bdc34597006d"
server_key_iv: "e5d5deb91456e56c1ce0733b2be73071"
server_key_version: "encryptionkey@0"
I20260811 22:35:22.555500 25908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:35:22.558151 25941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:22.559425 25908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:35:22.559573 25908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "6cb70d7beec04637a2a95edc0a396ab6"
format_stamp: "Formatted at 2026-08-11 22:35:22 on dist-test-slave-8hhm"
server_key: "b254807c6f2a11347ad1bdc34597006d"
server_key_iv: "e5d5deb91456e56c1ce0733b2be73071"
server_key_version: "encryptionkey@0"
I20260811 22:35:22.559698 25908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:22.588809 25908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:22.591909 25908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:22.592134 25908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:22.599610 25908 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:36793
I20260811 22:35:22.599632 26040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:36793 every 8 connection(s)
I20260811 22:35:22.600587 25908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:22.603406 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:22.606551 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 25908
I20260811 22:35:22.606724 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:22.607052 19278 external_mini_cluster.cc:1428] Setting key 987eaa5645003b1e50fb97e96fbd2a47
I20260811 22:35:22.609912 26041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6: Bootstrap starting.
I20260811 22:35:22.612169 26041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:22.612936 26041 log.cc:824] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:22.614873 26041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6: No bootstrap required, opened a new log
I20260811 22:35:22.616237 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:22.608552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38461 (local address 127.18.211.190:36793)
0811 22:35:22.609031 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:22.609046 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:22.609080 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:35:22.609713 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:22.609729 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:22.610121 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:22.610457 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:22.610888 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:22.611867 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:22.612819 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:22.613404 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:22.613579 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:22.614511 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:22.614519 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:22.614540 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:22.614932 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:22.615019 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:22.615052 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:22.615199 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:22.615387 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:35:22.615534 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":174,"threads_started":1}
I20260811 22:35:22.619130 26041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb70d7beec04637a2a95edc0a396ab6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36793 } }
I20260811 22:35:22.619444 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:22.619438 26041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:22.619531 26041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb70d7beec04637a2a95edc0a396ab6, State: Initialized, Role: FOLLOWER
I20260811 22:35:22.619951 26041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb70d7beec04637a2a95edc0a396ab6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36793 } }
I20260811 22:35:22.620105 26041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:22.620200 26041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:22.620321 26041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:22.621387 26041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb70d7beec04637a2a95edc0a396ab6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36793 } }
I20260811 22:35:22.621806 26041 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cb70d7beec04637a2a95edc0a396ab6; no voters:
I20260811 22:35:22.622187 26041 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:22.622309 26057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:22.622619 26057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [term 1 LEADER]: Becoming Leader. State: Replica: 6cb70d7beec04637a2a95edc0a396ab6, State: Running, Role: LEADER
I20260811 22:35:22.623010 26057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb70d7beec04637a2a95edc0a396ab6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36793 } }
I20260811 22:35:22.623553 26041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:22.624231 26057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb70d7beec04637a2a95edc0a396ab6. Latest consensus state: current_term: 1 leader_uuid: "6cb70d7beec04637a2a95edc0a396ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb70d7beec04637a2a95edc0a396ab6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36793 } } }
I20260811 22:35:22.624990 26057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:22.624869 26059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cb70d7beec04637a2a95edc0a396ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb70d7beec04637a2a95edc0a396ab6" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 36793 } } }
I20260811 22:35:22.625349 26059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:22.626917 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:22.621728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60287 (local address 127.18.211.190:36793)
0811 22:35:22.621859 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:22.621864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:22.622036 (+ 172us) server_negotiation.cc:407] Connection header received
0811 22:35:22.622140 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:22.622144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:22.622192 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:22.622296 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:22.622640 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:22.623389 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:22.624530 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:22.624940 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:22.625021 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:22.626250 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:22.626255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:22.626259 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:22.626275 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:22.626306 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:22.626319 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:22.626411 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:22.626760 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 22:35:22.626793 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 22:35:22.629205 26091 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:22.629331 26091 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:22.629406 26076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:22.630084 26076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:22.635882 26076 catalog_manager.cc:1347] Generated new cluster ID: fc59f752c23648ea9ecc655e819956e7
I20260811 22:35:22.635959 26076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:22.649405 26076 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:22.650416 26076 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:22.661760 26076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cb70d7beec04637a2a95edc0a396ab6: Generated new TSK 0
I20260811 22:35:22.662653 26076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:22.674876 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 25908
W20260811 22:35:22.692569 25881 connection.cc:535] client connection to 127.18.211.190:36793 recv error: Network error: failed to read from TLS socket (remote: 127.18.211.190:36793): Connection reset by peer (error 104)
2026-08-11T22:35:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1722 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:22 dist-test-slave-8hhm krb5kdc[26101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:22 dist-test-slave-8hhm krb5kdc[26101](info): set up 2 sockets
Aug 11 22:35:22 dist-test-slave-8hhm krb5kdc[26101](info): commencing operation
krb5kdc: starting...
W20260811 22:35:23.978765 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:24 dist-test-slave-8hhm krb5kdc[26101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:24.111658 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:37595
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:24.228192 26124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:24.228530 26124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:24.228622 26124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:24.232903 26124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:24.233016 26124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:24.233054 26124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:24.233115 26124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:24.233152 26124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:24.238678 26124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:37595
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:37595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:24.239893 26124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:24.240994 26124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:24.250171 26130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:24.250428 26124 server_base.cc:1034] running on GCE node
W20260811 22:35:24.250159 26131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:24.250159 26137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:24.251248 26124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:24.252463 26124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:24.253679 26124 hybrid_clock.cc:648] HybridClock initialized: now 1786487724253658 us; error 31 us; skew 500 ppm
Aug 11 22:35:24 dist-test-slave-8hhm krb5kdc[26101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487724, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:24.256696 26124 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:24.257705 26124 webserver.cc:460] Webserver started at http://127.18.211.190:39459/ using document root <none> and password file <none>
I20260811 22:35:24.258356 26124 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:24.258466 26124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:24.258725 26124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:24.260902 26124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "28d21f09b78944a0afc82fe7ac18b31b"
format_stamp: "Formatted at 2026-08-11 22:35:24 on dist-test-slave-8hhm"
server_key: "74647dc06fac634419216e15af0d64d3"
server_key_iv: "8795a8938b34f9f41b4f3a42bfa1a41e"
server_key_version: "encryptionkey@0"
I20260811 22:35:24.261524 26124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "28d21f09b78944a0afc82fe7ac18b31b"
format_stamp: "Formatted at 2026-08-11 22:35:24 on dist-test-slave-8hhm"
server_key: "74647dc06fac634419216e15af0d64d3"
server_key_iv: "8795a8938b34f9f41b4f3a42bfa1a41e"
server_key_version: "encryptionkey@0"
I20260811 22:35:24.265429 26124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:35:24.267961 26147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:24.269389 26124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:35:24.269536 26124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "28d21f09b78944a0afc82fe7ac18b31b"
format_stamp: "Formatted at 2026-08-11 22:35:24 on dist-test-slave-8hhm"
server_key: "74647dc06fac634419216e15af0d64d3"
server_key_iv: "8795a8938b34f9f41b4f3a42bfa1a41e"
server_key_version: "encryptionkey@0"
I20260811 22:35:24.269661 26124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:24.274381 26124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:24.277740 26124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:24.277932 26124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:24.285189 26124 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:37595
I20260811 22:35:24.285207 26247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:37595 every 8 connection(s)
I20260811 22:35:24.286221 26124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:24.288661 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26124
I20260811 22:35:24.288825 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:24.289119 19278 external_mini_cluster.cc:1428] Setting key 5e4e57ea4586496e330b443f85274ef9
I20260811 22:35:24.289113 26249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:24.295158 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:24.290623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44161 (local address 127.18.211.190:37595)
0811 22:35:24.291060 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:24.291081 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:24.291126 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:35:24.292066 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:24.292100 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:24.292490 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:24.292898 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:24.293687 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:24.293702 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:24.293717 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:24.293732 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:24.293799 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:24.293826 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:24.293970 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:24.294042 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 22:35:24.294209 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":136,"threads_started":1}
I20260811 22:35:24.296859 26249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b: Bootstrap starting.
I20260811 22:35:24.298139 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:24.299471 26249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:24.300271 26249 log.cc:824] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:24.301968 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:24.300213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43999 (local address 127.18.211.190:37595)
0811 22:35:24.300348 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:24.300352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:24.300574 (+ 222us) server_negotiation.cc:407] Connection header received
0811 22:35:24.300722 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:24.300725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:24.300784 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:24.300871 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:24.301568 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:24.301571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:24.301573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:24.301583 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:24.301608 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:24.301617 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:24.301687 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:24.301816 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 22:35:24.301845 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:35:24.302887 26249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b: No bootstrap required, opened a new log
I20260811 22:35:24.306598 26249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d21f09b78944a0afc82fe7ac18b31b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37595 } }
I20260811 22:35:24.306876 26249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:24.306965 26249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d21f09b78944a0afc82fe7ac18b31b, State: Initialized, Role: FOLLOWER
I20260811 22:35:24.307494 26249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d21f09b78944a0afc82fe7ac18b31b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37595 } }
I20260811 22:35:24.307672 26249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:24.307749 26249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:24.307879 26249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:24.308836 26249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d21f09b78944a0afc82fe7ac18b31b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37595 } }
I20260811 22:35:24.309288 26249 leader_election.cc:302] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28d21f09b78944a0afc82fe7ac18b31b; no voters:
I20260811 22:35:24.309654 26249 leader_election.cc:288] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:24.309782 26268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:24.310042 26268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [term 1 LEADER]: Becoming Leader. State: Replica: 28d21f09b78944a0afc82fe7ac18b31b, State: Running, Role: LEADER
I20260811 22:35:24.310396 26268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d21f09b78944a0afc82fe7ac18b31b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37595 } }
I20260811 22:35:24.311002 26249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:24.312247 26271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d21f09b78944a0afc82fe7ac18b31b. Latest consensus state: current_term: 1 leader_uuid: "28d21f09b78944a0afc82fe7ac18b31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d21f09b78944a0afc82fe7ac18b31b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37595 } } }
I20260811 22:35:24.312386 26271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:24.312726 26269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28d21f09b78944a0afc82fe7ac18b31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d21f09b78944a0afc82fe7ac18b31b" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 37595 } } }
I20260811 22:35:24.313427 26269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:24.313584 26290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:24.316419 26290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:24.322576 26290 catalog_manager.cc:1347] Generated new cluster ID: 5afc4ac4c6b04e11be685659b09a1d38
I20260811 22:35:24.322736 26290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:24.334800 26290 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:24.335170 26290 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:24.343364 26290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28d21f09b78944a0afc82fe7ac18b31b: Generated new TSK 0
I20260811 22:35:24.344126 26290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:24.349895 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26124
2026-08-11T22:35:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1678 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:24 dist-test-slave-8hhm krb5kdc[26311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:24 dist-test-slave-8hhm krb5kdc[26311](info): set up 2 sockets
Aug 11 22:35:24 dist-test-slave-8hhm krb5kdc[26311](info): commencing operation
krb5kdc: starting...
W20260811 22:35:25.681384 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:25 dist-test-slave-8hhm krb5kdc[26311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:25.814246 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42289
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:42467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:42289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:25.931005 26338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:25.931279 26338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:25.931321 26338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:25.935464 26338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:25.935532 26338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:25.935552 26338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:25.935570 26338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:25.935622 26338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:25.940996 26338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:42467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:42289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:25.942181 26338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:25.943281 26338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:25.952251 26346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:25.952263 26348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:25.952272 26350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:25.952661 26338 server_base.cc:1034] running on GCE node
I20260811 22:35:25.953423 26338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:25.954602 26338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:25.955801 26338 hybrid_clock.cc:648] HybridClock initialized: now 1786487725955786 us; error 25 us; skew 500 ppm
Aug 11 22:35:25 dist-test-slave-8hhm krb5kdc[26311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487725, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:25.965734 26338 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:25.966728 26338 webserver.cc:460] Webserver started at http://127.18.211.190:34253/ using document root <none> and password file <none>
I20260811 22:35:25.967326 26338 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:25.967425 26338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:25.967679 26338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:25.969961 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "eaa4885693a84a89af6b023f499cdc54"
format_stamp: "Formatted at 2026-08-11 22:35:25 on dist-test-slave-8hhm"
server_key: "d486760307fc2b049e607a1846074fab"
server_key_iv: "43bc61d25185858cf3b8cf0b5dec61ca"
server_key_version: "encryptionkey@0"
I20260811 22:35:25.970548 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "eaa4885693a84a89af6b023f499cdc54"
format_stamp: "Formatted at 2026-08-11 22:35:25 on dist-test-slave-8hhm"
server_key: "d486760307fc2b049e607a1846074fab"
server_key_iv: "43bc61d25185858cf3b8cf0b5dec61ca"
server_key_version: "encryptionkey@0"
I20260811 22:35:25.974340 26338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:35:25.976776 26365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:25.978057 26338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:35:25.978220 26338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "eaa4885693a84a89af6b023f499cdc54"
format_stamp: "Formatted at 2026-08-11 22:35:25 on dist-test-slave-8hhm"
server_key: "d486760307fc2b049e607a1846074fab"
server_key_iv: "43bc61d25185858cf3b8cf0b5dec61ca"
server_key_version: "encryptionkey@0"
I20260811 22:35:25.978332 26338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:25.996230 26338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:25.999053 26338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:25.999264 26338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:26.006438 26338 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:42289
I20260811 22:35:26.006461 26464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:42289 every 8 connection(s)
I20260811 22:35:26.007393 26338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:26.010152 26466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:26.012280 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26338
I20260811 22:35:26.012444 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:26.012758 19278 external_mini_cluster.cc:1428] Setting key feac5c292dd6012eb44a50326c2d6581
I20260811 22:35:26.016398 26466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54: Bootstrap starting.
I20260811 22:35:26.018646 26466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:26.019385 26466 log.cc:824] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:26.021618 26466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54: No bootstrap required, opened a new log
Aug 11 22:35:26 dist-test-slave-8hhm krb5kdc[26311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:26.026049 26466 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa4885693a84a89af6b023f499cdc54" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42289 } }
I20260811 22:35:26.026383 26466 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:26.026444 26466 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa4885693a84a89af6b023f499cdc54, State: Initialized, Role: FOLLOWER
I20260811 22:35:26.026990 26466 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa4885693a84a89af6b023f499cdc54" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42289 } }
I20260811 22:35:26.027175 26466 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:26.027237 26466 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:26.027328 26466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:26.028421 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:26.014305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60213 (local address 127.18.211.190:42289)
0811 22:35:26.014726 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:26.014741 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:26.014776 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:35:26.015537 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:26.015555 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:26.015900 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:26.016234 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:26.017079 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:26.018141 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:26.019069 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:26.019587 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:26.019764 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:26.022423 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:26.022430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:26.022446 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:26.022478 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:26.025081 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:26.025578 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:26.025585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:26.025591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:26.025747 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:26.026015 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:26.026019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:26.026021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:26.026566 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:26.026790 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:26.027098 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 22:35:26.027420 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":119,"threads_started":1}
I20260811 22:35:26.028503 26466 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa4885693a84a89af6b023f499cdc54" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42289 } }
I20260811 22:35:26.028851 26466 leader_election.cc:302] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaa4885693a84a89af6b023f499cdc54; no voters:
I20260811 22:35:26.029245 26466 leader_election.cc:288] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:26.029383 26473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:26.029616 26473 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [term 1 LEADER]: Becoming Leader. State: Replica: eaa4885693a84a89af6b023f499cdc54, State: Running, Role: LEADER
I20260811 22:35:26.029984 26473 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa4885693a84a89af6b023f499cdc54" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42289 } }
I20260811 22:35:26.030567 26466 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:26.031250 26476 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaa4885693a84a89af6b023f499cdc54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa4885693a84a89af6b023f499cdc54" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42289 } } }
I20260811 22:35:26.031370 26476 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:26.031612 26477 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaa4885693a84a89af6b023f499cdc54. Latest consensus state: current_term: 1 leader_uuid: "eaa4885693a84a89af6b023f499cdc54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa4885693a84a89af6b023f499cdc54" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42289 } } }
I20260811 22:35:26.031695 26477 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:26.031934 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:26.032137 26482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:26.035413 26482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:26.041481 26482 catalog_manager.cc:1347] Generated new cluster ID: 850b7d5077e6476ab3569a5d6b810a4d
I20260811 22:35:26.041546 26482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:26.043608 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:26.034308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48587 (local address 127.18.211.190:42289)
0811 22:35:26.034457 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:26.034462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:26.034834 (+ 372us) server_negotiation.cc:407] Connection header received
0811 22:35:26.034959 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:26.034962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:26.035010 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:26.035102 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:26.035966 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:26.036777 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:26.038034 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:26.038369 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:26.038481 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:26.040167 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:26.040171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:26.040174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:26.040199 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:26.041870 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:26.042374 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:26.042378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:26.042379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:26.042467 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:26.042773 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:26.042776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:26.042777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:26.042969 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:26.043091 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:26.043355 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 22:35:26.043465 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 22:35:26.048197 26482 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:26.049064 26482 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:26.060832 26482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaa4885693a84a89af6b023f499cdc54: Generated new TSK 0
I20260811 22:35:26.061621 26482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:26.065605 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26338
2026-08-11T22:35:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1719 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:26 dist-test-slave-8hhm krb5kdc[26520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:26 dist-test-slave-8hhm krb5kdc[26520](info): set up 2 sockets
Aug 11 22:35:26 dist-test-slave-8hhm krb5kdc[26520](info): commencing operation
krb5kdc: starting...
W20260811 22:35:27.384541 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:27 dist-test-slave-8hhm krb5kdc[26520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:27.517549 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42491
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:37573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:42491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:27.631062 26557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:27.631330 26557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:27.631389 26557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:27.635435 26557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:27.635495 26557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:27.635514 26557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:27.635530 26557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:27.635607 26557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:27.641289 26557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:42491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:42491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:27.642459 26557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:27.643544 26557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:27.652324 26580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:27.652315 26573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:27.652473 26557 server_base.cc:1034] running on GCE node
W20260811 22:35:27.652315 26575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:27.653196 26557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:27.654321 26557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:27.655550 26557 hybrid_clock.cc:648] HybridClock initialized: now 1786487727655530 us; error 30 us; skew 500 ppm
Aug 11 22:35:27 dist-test-slave-8hhm krb5kdc[26520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487727, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:27.658558 26557 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:27.659463 26557 webserver.cc:460] Webserver started at http://127.18.211.190:33563/ using document root <none> and password file <none>
I20260811 22:35:27.660035 26557 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:27.660105 26557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:27.660355 26557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:27.662423 26557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "dd67730276f241a698f9c86ecc02d2ff"
format_stamp: "Formatted at 2026-08-11 22:35:27 on dist-test-slave-8hhm"
server_key: "d18993edbf5e346e9c4e6db25889f170"
server_key_iv: "1e9e1405fa4899c6b46103de250920a7"
server_key_version: "encryptionkey@0"
I20260811 22:35:27.662985 26557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "dd67730276f241a698f9c86ecc02d2ff"
format_stamp: "Formatted at 2026-08-11 22:35:27 on dist-test-slave-8hhm"
server_key: "d18993edbf5e346e9c4e6db25889f170"
server_key_iv: "1e9e1405fa4899c6b46103de250920a7"
server_key_version: "encryptionkey@0"
I20260811 22:35:27.666667 26557 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:35:27.669036 26600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:27.670318 26557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:35:27.670447 26557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "dd67730276f241a698f9c86ecc02d2ff"
format_stamp: "Formatted at 2026-08-11 22:35:27 on dist-test-slave-8hhm"
server_key: "d18993edbf5e346e9c4e6db25889f170"
server_key_iv: "1e9e1405fa4899c6b46103de250920a7"
server_key_version: "encryptionkey@0"
I20260811 22:35:27.670553 26557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:27.683300 26557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:27.686571 26557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:27.686779 26557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:27.693885 26557 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:42491
I20260811 22:35:27.693907 26701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:42491 every 8 connection(s)
I20260811 22:35:27.694783 26557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:27.697583 26703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:27.703052 26703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff: Bootstrap starting.
I20260811 22:35:27.704487 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26557
I20260811 22:35:27.704653 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:27.704960 19278 external_mini_cluster.cc:1428] Setting key fba3b9c795741e44b664479872a3db5a
I20260811 22:35:27.705833 26703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:27.706748 26703 log.cc:824] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:27.708645 26703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff: No bootstrap required, opened a new log
I20260811 22:35:27.712044 26703 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd67730276f241a698f9c86ecc02d2ff" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42491 } }
I20260811 22:35:27.712263 26703 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:27.712325 26703 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd67730276f241a698f9c86ecc02d2ff, State: Initialized, Role: FOLLOWER
I20260811 22:35:27.712814 26703 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd67730276f241a698f9c86ecc02d2ff" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42491 } }
I20260811 22:35:27.713055 26703 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:27.713145 26703 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:27.713267 26703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:27.713891 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:27.706100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49411 (local address 127.18.211.190:42491)
0811 22:35:27.706526 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:27.706540 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:27.706574 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:35:27.707370 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:27.707394 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:27.707780 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:27.708114 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:27.708504 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:27.709581 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:27.710443 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:27.710965 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:27.711151 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:27.711962 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:27.711968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:27.711979 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:27.712446 (+ 467us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:27.712501 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:27.712524 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:27.712639 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:27.712810 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:35:27.713051 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":128,"threads_started":1}
I20260811 22:35:27.714212 26703 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd67730276f241a698f9c86ecc02d2ff" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42491 } }
I20260811 22:35:27.714555 26703 leader_election.cc:302] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd67730276f241a698f9c86ecc02d2ff; no voters:
I20260811 22:35:27.714911 26703 leader_election.cc:288] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:27.714975 26713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:27.715185 26713 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [term 1 LEADER]: Becoming Leader. State: Replica: dd67730276f241a698f9c86ecc02d2ff, State: Running, Role: LEADER
I20260811 22:35:27.715526 26713 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd67730276f241a698f9c86ecc02d2ff" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42491 } }
I20260811 22:35:27.716178 26703 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:27.716986 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:27.717200 26716 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd67730276f241a698f9c86ecc02d2ff. Latest consensus state: current_term: 1 leader_uuid: "dd67730276f241a698f9c86ecc02d2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd67730276f241a698f9c86ecc02d2ff" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42491 } } }
I20260811 22:35:27.717358 26716 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:27.717765 26736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:27.718860 26714 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd67730276f241a698f9c86ecc02d2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd67730276f241a698f9c86ecc02d2ff" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 42491 } } }
I20260811 22:35:27.718992 26714 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:27.721042 26736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:27.725157 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:27.719211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36249 (local address 127.18.211.190:42491)
0811 22:35:27.719361 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:27.719365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:27.721426 (+ 2061us) server_negotiation.cc:407] Connection header received
0811 22:35:27.721562 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:27.721567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:27.721613 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:27.721693 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:27.722016 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:27.722501 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:27.723362 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:27.723710 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:27.723825 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:27.724722 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:27.724727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:27.724729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:27.724742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:27.724770 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:27.724783 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:27.724862 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:27.724977 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 22:35:27.725027 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:35:27.727164 26736 catalog_manager.cc:1347] Generated new cluster ID: cfbe6ea140aa42db8fbe9101761da887
I20260811 22:35:27.727231 26736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:27.755299 26736 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:27.757371 26736 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:27.766539 26736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd67730276f241a698f9c86ecc02d2ff: Generated new TSK 0
I20260811 22:35:27.767314 26736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:27.776381 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26557
2026-08-11T22:35:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1701 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:27 dist-test-slave-8hhm krb5kdc[26768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:27 dist-test-slave-8hhm krb5kdc[26768](info): set up 2 sockets
Aug 11 22:35:27 dist-test-slave-8hhm krb5kdc[26768](info): commencing operation
krb5kdc: starting...
W20260811 22:35:29.063339 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:29 dist-test-slave-8hhm krb5kdc[26768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:29.196236 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:40225
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:40225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:29.310446 26794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:29.310714 26794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:29.310756 26794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:29.314656 26794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:29.314713 26794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:29.314731 26794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:29.314749 26794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:29.314764 26794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:29.319950 26794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:40225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:40225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:29.321023 26794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:29.322080 26794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:29.330906 26803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:29.330917 26805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:29.331180 26794 server_base.cc:1034] running on GCE node
W20260811 22:35:29.331346 26802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:29.332058 26794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:29.333191 26794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:29.334425 26794 hybrid_clock.cc:648] HybridClock initialized: now 1786487729334411 us; error 25 us; skew 500 ppm
Aug 11 22:35:29 dist-test-slave-8hhm krb5kdc[26768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:29.337298 26794 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:29.338179 26794 webserver.cc:460] Webserver started at http://127.18.211.190:39609/ using document root <none> and password file <none>
I20260811 22:35:29.338764 26794 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:29.338869 26794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:29.339120 26794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:29.341183 26794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "d4b954b3c542460781bd96da620f523a"
format_stamp: "Formatted at 2026-08-11 22:35:29 on dist-test-slave-8hhm"
server_key: "e18e957fe0e2f03aa3c9669eda378b38"
server_key_iv: "c81e9eb46cef395b9b3bb76941ac61ea"
server_key_version: "encryptionkey@0"
I20260811 22:35:29.341892 26794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "d4b954b3c542460781bd96da620f523a"
format_stamp: "Formatted at 2026-08-11 22:35:29 on dist-test-slave-8hhm"
server_key: "e18e957fe0e2f03aa3c9669eda378b38"
server_key_iv: "c81e9eb46cef395b9b3bb76941ac61ea"
server_key_version: "encryptionkey@0"
I20260811 22:35:29.345697 26794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:35:29.348116 26816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:29.349433 26794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:35:29.349591 26794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "d4b954b3c542460781bd96da620f523a"
format_stamp: "Formatted at 2026-08-11 22:35:29 on dist-test-slave-8hhm"
server_key: "e18e957fe0e2f03aa3c9669eda378b38"
server_key_iv: "c81e9eb46cef395b9b3bb76941ac61ea"
server_key_version: "encryptionkey@0"
I20260811 22:35:29.349704 26794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:29.354293 26794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:29.357098 26794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:29.357317 26794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:29.364544 26794 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:40225
I20260811 22:35:29.364579 26914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:40225 every 8 connection(s)
I20260811 22:35:29.365775 26794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:29.369287 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:29.373351 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26794
I20260811 22:35:29.373489 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:29.373746 19278 external_mini_cluster.cc:1428] Setting key cba4bf55cac8da1089e34cb4f01da112
I20260811 22:35:29.375432 26915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a: Bootstrap starting.
I20260811 22:35:29.378044 26915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:29.378855 26915 log.cc:824] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:29.378930 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:29.375107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51783 (local address 127.18.211.190:40225)
0811 22:35:29.375522 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:29.375538 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:29.375607 (+ 69us) server_negotiation.cc:407] Connection header received
0811 22:35:29.376248 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:29.376265 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:29.376547 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:29.376836 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:29.377707 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:29.377712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:29.377723 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:29.377733 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:29.377786 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:29.377804 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:29.377889 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:29.378055 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:35:29.378230 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":135,"threads_started":1}
I20260811 22:35:29.381021 26915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a: No bootstrap required, opened a new log
I20260811 22:35:29.381880 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:29.385026 26915 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b954b3c542460781bd96da620f523a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 40225 } }
I20260811 22:35:29.385334 26915 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:35:29.385406 26915 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b954b3c542460781bd96da620f523a, State: Initialized, Role: FOLLOWER
I20260811 22:35:29.385725 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:29.383960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60303 (local address 127.18.211.190:40225)
0811 22:35:29.384084 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:29.384088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:29.384293 (+ 205us) server_negotiation.cc:407] Connection header received
0811 22:35:29.384409 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:29.384412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:29.384448 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:29.384527 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:29.385278 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:29.385281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:29.385284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:35:29.385297 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:29.385324 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:35:29.385353 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:29.385427 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:29.385569 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 22:35:29.385608 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 22:35:29.385931 26915 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b954b3c542460781bd96da620f523a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 40225 } }
I20260811 22:35:29.386113 26915 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:35:29.386199 26915 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:35:29.386300 26915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:29.387285 26915 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b954b3c542460781bd96da620f523a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 40225 } }
I20260811 22:35:29.387635 26915 leader_election.cc:302] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4b954b3c542460781bd96da620f523a; no voters:
I20260811 22:35:29.387961 26915 leader_election.cc:288] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:35:29.388037 26937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:29.388264 26937 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [term 1 LEADER]: Becoming Leader. State: Replica: d4b954b3c542460781bd96da620f523a, State: Running, Role: LEADER
I20260811 22:35:29.388571 26937 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b954b3c542460781bd96da620f523a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 40225 } }
I20260811 22:35:29.389174 26915 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:29.390667 26940 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4b954b3c542460781bd96da620f523a. Latest consensus state: current_term: 1 leader_uuid: "d4b954b3c542460781bd96da620f523a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b954b3c542460781bd96da620f523a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 40225 } } }
I20260811 22:35:29.390820 26940 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:29.391089 26938 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4b954b3c542460781bd96da620f523a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b954b3c542460781bd96da620f523a" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 40225 } } }
I20260811 22:35:29.391185 26938 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:29.391202 26956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:29.394654 26956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:29.400122 26956 catalog_manager.cc:1347] Generated new cluster ID: b95ef4531ac546218d875732fa530924
I20260811 22:35:29.400243 26956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:29.422279 26956 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:29.422724 26956 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:29.427497 26956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4b954b3c542460781bd96da620f523a: Generated new TSK 0
I20260811 22:35:29.428227 26956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:35:29.439867 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26794
2026-08-11T22:35:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1659 ms)
[----------] 18 tests from AuthTokenIssuingTest (30840 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:29 dist-test-slave-8hhm krb5kdc[26974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:29 dist-test-slave-8hhm krb5kdc[26974](info): set up 2 sockets
Aug 11 22:35:29 dist-test-slave-8hhm krb5kdc[26974](info): commencing operation
krb5kdc: starting...
W20260811 22:35:30.746248 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:30 dist-test-slave-8hhm krb5kdc[26974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:30.879997 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:35131
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:34405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:35131,127.18.211.189:40609,127.18.211.188:34739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:30.999483 26997 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:35:30.999786 26997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:30.999810 26997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:30.999848 26997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:31.003772 26997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:31.003825 26997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:31.003844 26997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:31.003861 26997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:31.003876 26997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:31.009068 26997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:34405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:35131,127.18.211.189:40609,127.18.211.188:34739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:35131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:31.010190 26997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:31.011158 26997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:31.020041 27003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:31.020041 27004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:31.020097 27009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:31.020267 26997 server_base.cc:1034] running on GCE node
I20260811 22:35:31.020987 26997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:31.022173 26997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:31.023339 26997 hybrid_clock.cc:648] HybridClock initialized: now 1786487731023319 us; error 31 us; skew 500 ppm
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:31.028975 26997 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:31.030019 26997 webserver.cc:460] Webserver started at http://127.18.211.190:45209/ using document root <none> and password file <none>
I20260811 22:35:31.030617 26997 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:31.030714 26997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:31.030961 26997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:31.033082 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "3d281614b628448a8435a3190f161df3"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "ca26010154c1ed8c402ee53f00c5bafd"
server_key_iv: "237c8abc1853f23e6abcae0c0682c3c2"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.033741 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "3d281614b628448a8435a3190f161df3"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "ca26010154c1ed8c402ee53f00c5bafd"
server_key_iv: "237c8abc1853f23e6abcae0c0682c3c2"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.037554 26997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:35:31.040143 27023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:31.041538 26997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:35:31.041700 26997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "3d281614b628448a8435a3190f161df3"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "ca26010154c1ed8c402ee53f00c5bafd"
server_key_iv: "237c8abc1853f23e6abcae0c0682c3c2"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.041818 26997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:31.057658 26997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:31.060745 26997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:31.060957 26997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:31.068521 26997 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:35131
I20260811 22:35:31.068835 27125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:35131 every 8 connection(s)
I20260811 22:35:31.069628 26997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:31.069990 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 26997
I20260811 22:35:31.070205 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:31.070523 19278 external_mini_cluster.cc:1428] Setting key e00c2b2b7eebc7a66a04cf152aef90d7
I20260811 22:35:31.072671 27128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:31.078495 27128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.211.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.189@KRBTEST.COM" created.
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:31.096312 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.081458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:35131 (local address 127.0.0.1:40270)
0811 22:35:31.082099 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.082153 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.083114 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.085474 (+ 2360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.085479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.085810 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.086063 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.086085 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.087268 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.087271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.088072 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.088082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.088753 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.088757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.088892 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.089577 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.089595 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.091049 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.093012 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.093022 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.093033 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.093338 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.093683 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.093686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.093688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.093805 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.094474 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.094481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.094654 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.094845 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.095276 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":189,"spinlock_wait_cycles":44160,"thread_start_us":165,"threads_started":1}
I20260811 22:35:31.096316 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.081458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.18.211.190:35131)
0811 22:35:31.082046 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.082095 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.082264 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:35:31.083429 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.083435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.083679 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.083878 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.086223 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.087121 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.088191 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.088590 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.088775 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.091205 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.091212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.091225 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.091255 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.092875 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.093454 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.093459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.093461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.093578 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.093908 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.093911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.093913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.094347 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.094494 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.094924 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.095225 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":465,"thread_start_us":150,"threads_started":1}
I20260811 22:35:31.098943 27128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab.
Entry for principal kudu/127.18.211.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab.
I20260811 22:35:31.099872 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.099551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60898)
0811 22:35:31.099701 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.099789 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 22:35:31.100225 27034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
I20260811 22:35:31.100360 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:40609
--webserver_interface=127.18.211.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab
--principal=kudu/127.18.211.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:34405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:35131,127.18.211.189:40609,127.18.211.188:34739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:35:31.101984 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.101690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60904)
0811 22:35:31.101828 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.101892 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:35:31.102169 27128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:40609: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
I20260811 22:35:31.151496 27128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } attempt: 1
I20260811 22:35:31.152485 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.152174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60906)
0811 22:35:31.152337 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.152403 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:35:31.153323 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.153029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60908)
0811 22:35:31.153142 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.153200 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:35:31.153456 27128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:40609: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
W20260811 22:35:31.218717 27146 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:35:31.219066 27146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:31.219144 27146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:31.219226 27146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:35:31.221601 27128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } attempt: 2
I20260811 22:35:31.222463 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.222117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60910)
0811 22:35:31.222280 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.222361 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:35:31.223258 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.223014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60926)
0811 22:35:31.223136 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.223190 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:35:31.223393 27128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:40609: Network error: Client connection negotiation failed: client connection to 127.18.211.189:40609: connect: Connection refused (error 111)
W20260811 22:35:31.223570 27146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:31.223623 27146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:31.223642 27146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:31.223698 27146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:31.223735 27146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:31.229008 27146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:34405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:35131,127.18.211.189:40609,127.18.211.188:34739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab
--principal=kudu/127.18.211.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:40609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.211.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:31.230089 27146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:31.231179 27146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:31.239935 27165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:31.239988 27162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:31.240108 27146 server_base.cc:1034] running on GCE node
W20260811 22:35:31.240020 27160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:31.240821 27146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:31.242003 27146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:31.243189 27146 hybrid_clock.cc:648] HybridClock initialized: now 1786487731243153 us; error 44 us; skew 500 ppm
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:31.246392 27146 init.cc:377] Logged in from keytab as kudu/127.18.211.189@KRBTEST.COM (short username kudu)
I20260811 22:35:31.247354 27146 webserver.cc:460] Webserver started at http://127.18.211.189:34777/ using document root <none> and password file <none>
I20260811 22:35:31.247922 27146 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:31.248049 27146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:31.248291 27146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:31.250412 27146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data/instance:
uuid: "a8b8aad09e564b509b0b0158fdd5c2bd"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "f486936ece4361790cb7283d230dc024"
server_key_iv: "70fb65ef6afe4ae10c3b92543c76dd93"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.251026 27146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/wal/instance:
uuid: "a8b8aad09e564b509b0b0158fdd5c2bd"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "f486936ece4361790cb7283d230dc024"
server_key_iv: "70fb65ef6afe4ae10c3b92543c76dd93"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.254758 27146 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:35:31.257324 27179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:31.258651 27146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:35:31.258808 27146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/wal
uuid: "a8b8aad09e564b509b0b0158fdd5c2bd"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "f486936ece4361790cb7283d230dc024"
server_key_iv: "70fb65ef6afe4ae10c3b92543c76dd93"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.258926 27146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:31.270452 27146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:31.273314 27146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:31.273517 27146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:31.280530 27146 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.189:40609
I20260811 22:35:31.280812 27269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.189:40609 every 8 connection(s)
I20260811 22:35:31.281577 27146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
I20260811 22:35:31.284366 27270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:31.287935 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 27146
I20260811 22:35:31.288064 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-1/wal/instance
I20260811 22:35:31.288368 19278 external_mini_cluster.cc:1428] Setting key deacb944e4694b53269d02170927ea0e
I20260811 22:35:31.288338 27270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.211.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.188@KRBTEST.COM" created.
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:31.305140 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.290726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40276 (local address 127.18.211.190:35131)
0811 22:35:31.290876 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.290879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.291885 (+ 1006us) server_negotiation.cc:407] Connection header received
0811 22:35:31.292834 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.292837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.292902 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.293011 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.294350 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.294893 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.296059 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.296309 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.296427 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.301307 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.301313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.301320 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.301346 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.302801 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.303445 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.303448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.303449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.303532 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.303927 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.303929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.303930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.304107 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.304184 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.304906 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.305013 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:35:31.306442 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.291184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:35131 (local address 127.0.0.1:40276)
0811 22:35:31.291724 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.291768 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.292618 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.293199 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.293207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.293619 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.294182 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.294205 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.295047 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.295050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.295927 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.295936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.296473 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.296477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.296649 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.297339 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.297356 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.301102 (+ 3746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.302921 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.302927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.302942 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.303334 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.303632 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.303638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.303643 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.303831 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.304218 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.304227 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.304601 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.304862 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.305240 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":152,"threads_started":1}
I20260811 22:35:31.308022 27270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
I20260811 22:35:31.314555 27128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab.
Entry for principal kudu/127.18.211.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab.
I20260811 22:35:31.317943 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:34739
--webserver_interface=127.18.211.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab
--principal=kudu/127.18.211.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:34405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:35131,127.18.211.189:40609,127.18.211.188:34739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
I20260811 22:35:31.320147 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.308562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60938)
0811 22:35:31.308679 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.308708 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.308785 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.309506 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.309509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.309634 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.309720 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.309725 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.310741 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.310744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.311257 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.311265 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.311860 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.311863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.312050 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.312519 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.312531 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.313905 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.316289 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.316292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.316294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.316557 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.316849 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.316852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.316854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.316951 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.319589 (+ 2638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.319595 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.319695 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.319827 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.319963 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":64}
I20260811 22:35:31.320664 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.308761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60938 (local address 127.18.211.189:40609)
0811 22:35:31.309093 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.309101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.309120 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:35:31.309154 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.309158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.309366 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.309470 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.309821 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.310578 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.311381 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.311745 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.311864 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.314033 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.314042 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.314052 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.314073 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.316163 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.316668 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.316673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.316674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.316751 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.317052 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.317055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.317056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.317349 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.317485 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.319951 (+ 2466us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.320122 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":229,"thread_start_us":148,"threads_started":1}
I20260811 22:35:31.323024 27270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:35:31.323748 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.323484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:56190)
0811 22:35:31.323595 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.323678 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:35:31.323975 27180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
I20260811 22:35:31.324538 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.314989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:60946)
0811 22:35:31.315131 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.315147 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.315244 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.315632 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.315634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.315768 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.316330 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.316347 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.317424 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.317427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.317974 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.317982 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.318446 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.318450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.318574 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.319017 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.319028 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.320479 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.322654 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.322660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.322665 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.322953 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.323318 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.323323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.323326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.323431 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.323968 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.323974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.324099 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.324232 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.324383 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 22:35:31.325095 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.315088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60946 (local address 127.18.211.189:40609)
0811 22:35:31.315395 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.315404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.315417 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:35:31.315455 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.315460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.315507 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.315603 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.316454 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.317292 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.318103 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.318336 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.318441 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.320637 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.320642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.320646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.320679 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.322491 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.323091 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.323096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.323100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.323197 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.323582 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.323589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.323593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.323864 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.323977 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.324314 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.324449 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":84,"threads_started":1}
I20260811 22:35:31.325201 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.324950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:56204)
0811 22:35:31.325057 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.325103 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:35:31.325549 27270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:34739: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
I20260811 22:35:31.325764 27128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:35:31.326395 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.326158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:56206)
0811 22:35:31.326276 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.326328 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:35:31.327049 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.326808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:56218)
0811 22:35:31.326949 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.326996 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 22:35:31.327200 27128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:34739: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
I20260811 22:35:31.358376 27128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } attempt: 1
I20260811 22:35:31.359246 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.358894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58828)
0811 22:35:31.359068 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.359147 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 22:35:31.360001 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.359764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58832)
0811 22:35:31.359881 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.359935 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:35:31.360147 27128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:34739: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
I20260811 22:35:31.374828 27270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } attempt: 1
I20260811 22:35:31.375761 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.375412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58836)
0811 22:35:31.375610 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.375669 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 22:35:31.376519 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.376284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58848)
0811 22:35:31.376396 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.376452 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:35:31.376696 27270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:34739: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
I20260811 22:35:31.435341 27128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } attempt: 2
W20260811 22:35:31.435352 27284 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:35:31.435721 27284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:31.435777 27284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:31.435853 27284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:35:31.436266 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.435960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58860)
0811 22:35:31.436119 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.436186 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:35:31.436970 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.436743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58864)
0811 22:35:31.436857 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.436909 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:35:31.437129 27128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:34739: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
W20260811 22:35:31.440138 27284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:31.440217 27284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:31.440263 27284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:31.440316 27284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:31.440352 27284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:31.444861 27270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } attempt: 2
I20260811 22:35:31.445647 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.445350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58872)
0811 22:35:31.445504 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.445567 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:35:31.445902 27284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:34405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:35131,127.18.211.189:40609,127.18.211.188:34739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab
--principal=kudu/127.18.211.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:34739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.211.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:31.446419 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.446206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58888)
0811 22:35:31.446301 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.446359 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 22:35:31.446609 27270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:34739: Network error: Client connection negotiation failed: client connection to 127.18.211.188:34739: connect: Connection refused (error 111)
I20260811 22:35:31.447099 27284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:31.448098 27284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:31.456621 27297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:31.456624 27300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:31.456621 27310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:31.456830 27284 server_base.cc:1034] running on GCE node
I20260811 22:35:31.457530 27284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:31.458616 27284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:31.459790 27284 hybrid_clock.cc:648] HybridClock initialized: now 1786487731459774 us; error 26 us; skew 500 ppm
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:31.462836 27284 init.cc:377] Logged in from keytab as kudu/127.18.211.188@KRBTEST.COM (short username kudu)
I20260811 22:35:31.463783 27284 webserver.cc:460] Webserver started at http://127.18.211.188:36451/ using document root <none> and password file <none>
I20260811 22:35:31.464310 27284 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:31.464370 27284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:31.464543 27284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:31.466722 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data/instance:
uuid: "fe3504b1f8604d6193400c722fb21dd3"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "02a17f199673a6259b0ea7ce99d0c055"
server_key_iv: "bf343ea052a3434ab4938b835bef1d6b"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.467243 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/wal/instance:
uuid: "fe3504b1f8604d6193400c722fb21dd3"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "02a17f199673a6259b0ea7ce99d0c055"
server_key_iv: "bf343ea052a3434ab4938b835bef1d6b"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.470920 27284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 22:35:31.473400 27321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:31.474612 27284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 22:35:31.474781 27284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/wal
uuid: "fe3504b1f8604d6193400c722fb21dd3"
format_stamp: "Formatted at 2026-08-11 22:35:31 on dist-test-slave-8hhm"
server_key: "02a17f199673a6259b0ea7ce99d0c055"
server_key_iv: "bf343ea052a3434ab4938b835bef1d6b"
server_key_version: "encryptionkey@0"
I20260811 22:35:31.474896 27284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:31.489642 27284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:31.493000 27284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:31.493256 27284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:31.500699 27284 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.188:34739
I20260811 22:35:31.501044 27419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.188:34739 every 8 connection(s)
I20260811 22:35:31.501784 27284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
I20260811 22:35:31.504819 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:31.505708 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 27284
I20260811 22:35:31.505862 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487433619239-19278-0/minicluster-data/master-2/wal/instance
I20260811 22:35:31.506246 19278 external_mini_cluster.cc:1428] Setting key 288b5533bc598c0fb1248de4b3faea7f
I20260811 22:35:31.510679 27422 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:31.522074 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.507637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.18.211.188:34739)
0811 22:35:31.508048 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.508064 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.508100 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:35:31.508799 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.508816 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.509149 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.509506 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.510570 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.511646 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.512892 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.513713 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.513905 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.516368 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.516374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.516384 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.516411 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.518382 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.518892 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.518897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.518902 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.519062 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.519418 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.519422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.519424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.520041 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.520252 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.520569 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.520878 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":132,"threads_started":1}
I20260811 22:35:31.522535 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.512521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:35131 (local address 127.0.0.1:56852)
0811 22:35:31.512957 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.513009 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.513145 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.513489 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.513495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.514063 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.514208 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.514214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.515010 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.515015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.515718 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.515727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.516280 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.516284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.516402 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.516916 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.516929 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.518577 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.520716 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.520730 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.520734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.520972 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.521339 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.521341 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.521342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.521439 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.521942 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.521947 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.522041 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.522175 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.522366 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260811 22:35:31.522688 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.512664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.18.211.190:35131)
0811 22:35:31.512816 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.512821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.513104 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:35:31.513279 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.513281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.513326 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.513396 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.514346 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.514878 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.515856 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.516110 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.516234 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.518724 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.518730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.518735 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.518765 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.520604 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.521077 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.521081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.521083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.521185 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.521538 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.521542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.521544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.521808 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.521931 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.522374 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.522509 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:35:31.524156 27422 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:31.533913 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.524716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:40609 (local address 127.0.0.1:57866)
0811 22:35:31.524817 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.524831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.524909 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.525468 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.525471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.525655 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.525744 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.525749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.526607 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.526612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.527189 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.527197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.527817 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.527821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.527930 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.528354 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.528376 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.530058 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.532217 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.532221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.532222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.532461 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.532794 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.532797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.532799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.532893 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.533405 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.533408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.533502 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.533608 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.533738 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 22:35:31.534075 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.524987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.18.211.189:40609)
0811 22:35:31.525175 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.525179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.525192 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:35:31.525292 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.525295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.525352 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.525451 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.525878 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.526478 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.527323 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.527656 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.527779 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.530244 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.530250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.530252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.530287 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.532086 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.532581 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.532585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.532587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.532687 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.533002 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.533006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.533008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.533309 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.533393 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.533688 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.533833 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 22:35:31.534742 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.524730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.18.211.190:35131)
0811 22:35:31.524960 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.524964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.524978 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:35:31.525013 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.525021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.525055 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.525143 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.525762 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.526241 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.527010 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.527226 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.527332 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.530780 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.530789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.530791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.530831 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.532233 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.532794 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.532798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.532799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.532885 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.533207 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.533209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.533256 (+ 47us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.533510 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.533772 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.533929 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.534562 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260811 22:35:31.534893 27422 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:35:31.536280 27128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } attempt: 3
I20260811 22:35:31.537811 27270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } attempt: 3
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Aug 11 22:35:31 dist-test-slave-8hhm krb5kdc[26974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487731, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
I20260811 22:35:31.545017 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.535425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58908)
0811 22:35:31.535515 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.535530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.535602 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.535860 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.535866 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.536037 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.536139 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.536145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.537810 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.537813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.538605 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.538615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.539151 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.539156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.539251 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.539794 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.539809 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.541277 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.543425 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.543427 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.543429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.543613 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.543893 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.543896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.543897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.543977 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.544410 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.544417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.544543 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.544666 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.544798 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 22:35:31.545120 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.535487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.18.211.188:34739)
0811 22:35:31.535600 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.535605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.535619 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:35:31.535675 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.535678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.535720 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.535803 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.536495 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.537699 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.538723 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.539034 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.539466 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.541399 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.541402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.541405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.541424 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.543287 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.543707 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.543712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.543716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.543795 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.544067 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.544070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.544071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.544308 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.544411 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.544752 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.544889 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:35:31.548715 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.536127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.18.211.189:40609)
0811 22:35:31.536240 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.536245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.536260 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:35:31.536330 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.536333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.536371 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.536446 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.537419 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.538092 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.539263 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.539850 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.539929 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.544391 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.544394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.544397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.544425 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.546364 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.547178 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.547182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.547185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.547288 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.547647 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.547652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.547655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.547872 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.548050 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.548430 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.548565 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:35:31.549403 27422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: Bootstrap starting.
I20260811 22:35:31.551573 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.536949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58912 (local address 127.18.211.188:34739)
0811 22:35:31.537318 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.537326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.537339 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:35:31.537371 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.537373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.537409 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.537972 (+ 563us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.538054 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.538785 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.540226 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.540726 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.540842 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.546418 (+ 5576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.546422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.546425 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.546451 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.549264 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.550006 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.550009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.550011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.550100 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.550424 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.550427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.550429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.550643 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.550748 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.551261 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.551378 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":302,"thread_start_us":126,"threads_started":1}
I20260811 22:35:31.551789 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.536675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58912)
0811 22:35:31.536834 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.536852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.536939 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.537543 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.537546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.537725 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.537851 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.537858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.538878 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.538882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.540091 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.540101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.541029 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.541035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.541159 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.541706 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.541725 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.545311 (+ 3586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.549430 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.549434 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.549436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.549871 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.550216 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.550219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.550221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.550315 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.550783 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.550789 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.550968 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.551210 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.551453 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:35:31.552747 27422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:31.553884 27422 log.cc:824] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:31.555913 27128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3: Bootstrap starting.
I20260811 22:35:31.556370 27422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: No bootstrap required, opened a new log
I20260811 22:35:31.558660 27128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:31.559238 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.538152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:34739 (local address 127.0.0.1:58920)
0811 22:35:31.538315 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:31.538329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:31.538398 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:31.539217 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:31.539220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:31.539421 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:31.539537 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.539547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.540754 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.540758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.541591 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:31.541605 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.542383 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.542386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:31.542470 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.543005 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:31.543022 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:31.544827 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:31.547708 (+ 2881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.547711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.547713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.547931 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.553110 (+ 5179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:31.553116 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:31.553119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:31.553314 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:31.558680 (+ 5366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:31.558683 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:31.558793 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 22:35:31.558922 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 22:35:31.559079 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 22:35:31.559255 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.538601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58920 (local address 127.18.211.188:34739)
0811 22:35:31.538980 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.538989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.538999 (+ 10us) server_negotiation.cc:407] Connection header received
0811 22:35:31.539029 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.539031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.539076 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.539651 (+ 575us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.539816 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.540593 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.541763 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.542222 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.542377 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.545385 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.545388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.545390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.545417 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.547591 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.548046 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.548049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.548050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.553008 (+ 4958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.558351 (+ 5343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.558357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.558362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.558567 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.558686 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.558978 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.559103 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":86,"threads_started":1}
I20260811 22:35:31.559548 27128 log.cc:824] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:31.561561 27422 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.561830 27422 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3504b1f8604d6193400c722fb21dd3, State: Initialized, Role: FOLLOWER
I20260811 22:35:31.562227 27422 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.562608 27270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: Bootstrap starting.
I20260811 22:35:31.564005 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.564146 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:31.564965 27422 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:31.567018 27270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:31.568045 27270 log.cc:824] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:31.569005 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 22:35:31.570793 27455 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:31.570879 27455 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:31.569697 27284 master_runner.cc:186] Error getting master registration for 127.18.211.190:35131: OK, instance_id { permanent_uuid: "3d281614b628448a8435a3190f161df3" instance_seqno: 1786487731066459 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:35:31.572849 27284 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:35:31.573422 27079 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d281614b628448a8435a3190f161df3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56868
I20260811 22:35:31.575026 27270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: No bootstrap required, opened a new log
I20260811 22:35:31.575089 27128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3: No bootstrap required, opened a new log
I20260811 22:35:31.578423 27128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.578681 27128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d281614b628448a8435a3190f161df3, State: Initialized, Role: FOLLOWER
I20260811 22:35:31.579053 27128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.579294 27270 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.579522 27270 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8b8aad09e564b509b0b0158fdd5c2bd, State: Initialized, Role: FOLLOWER
I20260811 22:35:31.579849 27270 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.579825 27465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.580188 27465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:31.580541 27128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:31.580629 27079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:35:31.580727 27079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:31.581135 27270 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:31.582219 27079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.583293 27079 leader_election.cc:288] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [CANDIDATE]: Term 1 election: Requested vote from peers a8b8aad09e564b509b0b0158fdd5c2bd (127.18.211.189:40609), fe3504b1f8604d6193400c722fb21dd3 (127.18.211.188:34739)
I20260811 22:35:31.584345 27376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d281614b628448a8435a3190f161df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fe3504b1f8604d6193400c722fb21dd3"
I20260811 22:35:31.584614 27376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:31.585099 27228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d281614b628448a8435a3190f161df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd"
I20260811 22:35:31.585338 27228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:31.586174 27376 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d281614b628448a8435a3190f161df3 in term 1.
I20260811 22:35:31.586644 27030 leader_election.cc:302] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d281614b628448a8435a3190f161df3, fe3504b1f8604d6193400c722fb21dd3; no voters:
W20260811 22:35:31.588054 27515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:31.588121 27515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:31.589560 27228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d281614b628448a8435a3190f161df3 in term 1.
I20260811 22:35:31.590363 27465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:31.590622 27465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [term 1 LEADER]: Becoming Leader. State: Replica: 3d281614b628448a8435a3190f161df3, State: Running, Role: LEADER
I20260811 22:35:31.590966 27465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } }
I20260811 22:35:31.593624 27465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d281614b628448a8435a3190f161df3. Latest consensus state: current_term: 1 leader_uuid: "3d281614b628448a8435a3190f161df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.593765 27465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:31.594791 26997 master_runner.cc:263] No leader master found from master 3d281614b628448a8435a3190f161df3
I20260811 22:35:31.594861 26997 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:35:31.597097 27502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:31.597173 27502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:31.597378 27466 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.597505 27466 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:31.597746 27146 master_runner.cc:263] No leader master found from master a8b8aad09e564b509b0b0158fdd5c2bd
I20260811 22:35:31.597811 27146 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:35:31.606076 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.588401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57876 (local address 127.18.211.189:40609)
0811 22:35:31.588538 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.588543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.595520 (+ 6977us) server_negotiation.cc:407] Connection header received
0811 22:35:31.595774 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.595778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.595829 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.595891 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.597244 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.598046 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.598965 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.599250 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.599356 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.601402 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.601409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.601414 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.601444 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.603741 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.604308 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.604313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.604316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.604424 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.604822 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.604828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.604830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.605128 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.605287 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.605752 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.605884 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:35:31.609495 27529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:31.609982 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.590363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58926 (local address 127.18.211.188:34739)
0811 22:35:31.590490 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.590495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.597804 (+ 7309us) server_negotiation.cc:407] Connection header received
0811 22:35:31.598096 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.598100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.598149 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.598229 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.599256 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.600097 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.601432 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.601851 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.601955 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.603909 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.603913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.603917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.603949 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.606038 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.606618 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.606622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.606624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.606725 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.608237 (+ 1512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.608241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.608243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.608468 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.609285 (+ 817us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.609681 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.609814 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:35:31.610569 27529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:31.616838 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:31.593911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.18.211.190:35131)
0811 22:35:31.594069 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.594074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.606904 (+ 12830us) server_negotiation.cc:407] Connection header received
0811 22:35:31.607107 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.607111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.607167 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.607227 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.608098 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.608980 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.610577 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:31.610913 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:31.611000 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:31.612825 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:31.612829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:31.612832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:31.612861 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:31.614776 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.615300 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.615304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.615306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.615403 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:31.615771 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:31.615775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:31.615777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:31.616023 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:31.616133 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:31.616549 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 22:35:31.616682 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:35:31.617152 27376 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [term 1 FOLLOWER]: Refusing update from remote peer 3d281614b628448a8435a3190f161df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:35:31.617940 27527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:35:31.622203 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d281614b628448a8435a3190f161df3. Latest consensus state: current_term: 1 leader_uuid: "3d281614b628448a8435a3190f161df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.622344 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:31.623170 27228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [term 1 FOLLOWER]: Refusing update from remote peer 3d281614b628448a8435a3190f161df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:35:31.623806 27527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:35:31.628939 27466 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d281614b628448a8435a3190f161df3. Latest consensus state: current_term: 1 leader_uuid: "3d281614b628448a8435a3190f161df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.629076 27466 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:31.633322 27465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d281614b628448a8435a3190f161df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.633442 27465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:31.633926 27534 mvcc.cc:205] Tried to move back new op lower bound from 7317453748694802432 to 7317453748601987072. Current Snapshot: MvccSnapshot[applied={T|T < 7317453748694802432}]
I20260811 22:35:31.634063 27465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d281614b628448a8435a3190f161df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.634199 27465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:31.634384 27466 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d281614b628448a8435a3190f161df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.634506 27466 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:31.635582 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d281614b628448a8435a3190f161df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d281614b628448a8435a3190f161df3" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 35131 } } peers { permanent_uuid: "a8b8aad09e564b509b0b0158fdd5c2bd" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 40609 } } peers { permanent_uuid: "fe3504b1f8604d6193400c722fb21dd3" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 34739 } } }
I20260811 22:35:31.635696 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:31.636150 27542 mvcc.cc:205] Tried to move back new op lower bound from 7317453748694802432 to 7317453748601987072. Current Snapshot: MvccSnapshot[applied={T|T < 7317453748694802432}]
I20260811 22:35:31.638561 27529 catalog_manager.cc:1347] Generated new cluster ID: 6675a9efed3a49fd9c0e70001e049332
I20260811 22:35:31.638631 27529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:31.670861 27529 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:31.672233 27529 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:31.702301 27529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d281614b628448a8435a3190f161df3: Generated new TSK 0
I20260811 22:35:31.703266 27529 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:35:31.735842 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.732458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57884 (local address 127.18.211.189:40609)
0811 22:35:31.732593 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.732598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.734693 (+ 2095us) server_negotiation.cc:407] Connection header received
0811 22:35:31.734908 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.734911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.734963 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.735030 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.735700 (+ 670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57884: BlockingRecv error: recv got EOF from 127.0.0.1:57884 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 22:35:31.737509 27135 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.733300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56890 (local address 127.18.211.190:35131)
0811 22:35:31.733421 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.733425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.736531 (+ 3106us) server_negotiation.cc:407] Connection header received
0811 22:35:31.736705 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.736709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.736757 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.736807 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.737349 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56890: BlockingRecv error: recv got EOF from 127.0.0.1:56890 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 22:35:31.738478 27431 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.734056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58938 (local address 127.18.211.188:34739)
0811 22:35:31.734174 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.734178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.737453 (+ 3275us) server_negotiation.cc:407] Connection header received
0811 22:35:31.737675 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.737680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.737737 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.737792 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.738348 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58938: BlockingRecv error: recv got EOF from 127.0.0.1:58938 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:35:31.754714 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.745676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.18.211.189:40609)
0811 22:35:31.745793 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.745797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.753582 (+ 7785us) server_negotiation.cc:407] Connection header received
0811 22:35:31.753843 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.753846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.753927 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.753982 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.754588 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57896: BlockingRecv error: recv got EOF from 127.0.0.1:57896 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:35:31.760020 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.758493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.18.211.189:40609)
0811 22:35:31.758594 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.758598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.759138 (+ 540us) server_negotiation.cc:407] Connection header received
0811 22:35:31.759322 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.759325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.759374 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.759423 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.759921 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57910: BlockingRecv error: recv got EOF from 127.0.0.1:57910 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 22:35:31.765681 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.764122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57920 (local address 127.18.211.189:40609)
0811 22:35:31.764214 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.764218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.764770 (+ 552us) server_negotiation.cc:407] Connection header received
0811 22:35:31.764947 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.764950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.764996 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.765042 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.765582 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57920: BlockingRecv error: recv got EOF from 127.0.0.1:57920 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 22:35:31.773505 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.771866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.18.211.189:40609)
0811 22:35:31.772003 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.772007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.772488 (+ 481us) server_negotiation.cc:407] Connection header received
0811 22:35:31.772674 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.772678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.772728 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.772780 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.773408 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57934: BlockingRecv error: recv got EOF from 127.0.0.1:57934 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:35:31.791714 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.789834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57940 (local address 127.18.211.189:40609)
0811 22:35:31.789987 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.789992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.790622 (+ 630us) server_negotiation.cc:407] Connection header received
0811 22:35:31.790791 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.790795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.790865 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.790977 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.791562 (+ 585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57940: BlockingRecv error: recv got EOF from 127.0.0.1:57940 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 22:35:31.811532 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.810188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.18.211.189:40609)
0811 22:35:31.810302 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.810307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.810613 (+ 306us) server_negotiation.cc:407] Connection header received
0811 22:35:31.810743 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.810746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.810812 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.810915 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.811397 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57952: BlockingRecv error: recv got EOF from 127.0.0.1:57952 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 22:35:31.846742 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.845589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.18.211.189:40609)
0811 22:35:31.845768 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.845772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.845886 (+ 114us) server_negotiation.cc:407] Connection header received
0811 22:35:31.846005 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.846007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.846056 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.846131 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.846618 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57956: BlockingRecv error: recv got EOF from 127.0.0.1:57956 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 22:35:31.914343 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:31.913086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.18.211.189:40609)
0811 22:35:31.913266 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:31.913271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:31.913463 (+ 192us) server_negotiation.cc:407] Connection header received
0811 22:35:31.913590 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:31.913593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:31.913644 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:31.913719 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:31.914220 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57964: BlockingRecv error: recv got EOF from 127.0.0.1:57964 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:35:32.046205 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:32.044637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.18.211.189:40609)
0811 22:35:32.044834 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:32.044839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:32.045000 (+ 161us) server_negotiation.cc:407] Connection header received
0811 22:35:32.045276 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:32.045281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:32.045351 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:32.045456 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:32.046071 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57976: BlockingRecv error: recv got EOF from 127.0.0.1:57976 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 22:35:32.305553 27283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:32.304472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57986 (local address 127.18.211.189:40609)
0811 22:35:32.304606 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:32.304609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:32.304627 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:35:32.304812 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:32.304814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:32.304861 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:32.304940 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:32.305453 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57986: BlockingRecv error: recv got EOF from 127.0.0.1:57986 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:35:32.572247 27455 catalog_manager.cc:1259] Loaded cluster ID: 6675a9efed3a49fd9c0e70001e049332
I20260811 22:35:32.572337 27455 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: loading cluster ID for follower catalog manager: success
I20260811 22:35:32.573987 27455 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: acquiring CA information for follower catalog manager: success
I20260811 22:35:32.575120 27455 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe3504b1f8604d6193400c722fb21dd3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:35:32.589504 27515 catalog_manager.cc:1259] Loaded cluster ID: 6675a9efed3a49fd9c0e70001e049332
I20260811 22:35:32.589576 27515 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: loading cluster ID for follower catalog manager: success
I20260811 22:35:32.591150 27515 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: acquiring CA information for follower catalog manager: success
I20260811 22:35:32.592296 27515 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a8b8aad09e564b509b0b0158fdd5c2bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:35:32.824501 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:32.820015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.18.211.189:40609)
0811 22:35:32.820314 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:32.820321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:32.820346 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:35:32.820568 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:32.820571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:32.820709 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:32.820845 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:32.821181 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.821829 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.822789 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.823066 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.823191 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:32.823560 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:32.824033 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:32.824186 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:32.824254 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 22:35:32.824317 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
I20260811 22:35:32.831372 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:32.827818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.18.211.188:34739)
0811 22:35:32.828033 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:32.828036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:32.828052 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:35:32.828135 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:32.828138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:32.828261 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:32.828364 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:32.828572 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.829063 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.830002 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.830230 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.830309 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:32.830592 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:32.831035 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:32.831129 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:32.831207 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 22:35:32.831253 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
I20260811 22:35:32.839295 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:32.834977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.18.211.188:34739)
0811 22:35:32.835115 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:32.835122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:32.835146 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:35:32.835300 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:32.835305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:32.835359 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:32.835435 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:32.835722 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.836474 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.837566 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.837782 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.837897 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:32.838792 (+ 895us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:32.838875 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:32.838941 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:32.839102 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:35:32.839150 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:35:32.839489 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:32.834662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58014 (local address 127.18.211.189:40609)
0811 22:35:32.834885 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:32.834892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:32.835512 (+ 620us) server_negotiation.cc:407] Connection header received
0811 22:35:32.835692 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:32.835696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:32.835747 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:32.835816 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:32.836162 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.836768 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.838247 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.838581 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.838703 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:32.838938 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:32.839019 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:32.839088 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:32.839291 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:35:32.839359 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 22:35:32.840096 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:32.834663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.18.211.190:35131)
0811 22:35:32.835134 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:32.835141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:32.835158 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:35:32.835393 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:32.835395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:32.835583 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:32.835702 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:32.835957 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.836711 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.838254 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:32.838489 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:32.838604 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:32.838941 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:32.839300 (+ 359us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:32.839712 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:32.839892 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 22:35:32.839955 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":77,"threads_started":1}
I20260811 22:35:32.843022 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 26997
I20260811 22:35:32.864759 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 27146
I20260811 22:35:32.881642 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 27284
2026-08-11T22:35:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3446 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:35:32 dist-test-slave-8hhm krb5kdc[27706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:35:32 dist-test-slave-8hhm krb5kdc[27706](info): set up 2 sockets
Aug 11 22:35:32 dist-test-slave-8hhm krb5kdc[27706](info): commencing operation
krb5kdc: starting...
W20260811 22:35:34.187888 19278 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/test-user.keytab.
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:35:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:35:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.211.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
Entry for principal kudu/127.18.211.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab.
I20260811 22:35:34.326522 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:46601
--webserver_interface=127.18.211.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:46601,127.18.211.189:41285,127.18.211.188:37259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:35:34.445421 27733 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:35:34.445827 27733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:34.445916 27733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:34.446038 27733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:34.450309 27733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:34.450428 27733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:34.450472 27733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:34.450516 27733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:34.450573 27733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:34.456436 27733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:46601,127.18.211.189:41285,127.18.211.188:37259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.190.keytab
--principal=kudu/127.18.211.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.190:46601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.211.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:34.457687 27733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:34.458926 27733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:34.468007 27747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:34.468042 27751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:34.468196 27733 server_base.cc:1034] running on GCE node
W20260811 22:35:34.468205 27749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:34.469071 27733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:34.470325 27733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:34.471577 27733 hybrid_clock.cc:648] HybridClock initialized: now 1786487734471560 us; error 40 us; skew 500 ppm
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:34.474678 27733 init.cc:377] Logged in from keytab as kudu/127.18.211.190@KRBTEST.COM (short username kudu)
I20260811 22:35:34.475631 27733 webserver.cc:460] Webserver started at http://127.18.211.190:34179/ using document root <none> and password file <none>
I20260811 22:35:34.476166 27733 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:34.476238 27733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:34.476409 27733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:34.478513 27733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data/instance:
uuid: "0a0edc261e1547588d877ec687311182"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "a1ffc05bcde375cbbe3adfacecd8d53a"
server_key_iv: "c65c24872041c5b7b252f54f8b5e5fcf"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.479049 27733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/wal/instance:
uuid: "0a0edc261e1547588d877ec687311182"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "a1ffc05bcde375cbbe3adfacecd8d53a"
server_key_iv: "c65c24872041c5b7b252f54f8b5e5fcf"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.482790 27733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:35:34.485360 27763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:34.486577 27733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:35:34.486687 27733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/wal
uuid: "0a0edc261e1547588d877ec687311182"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "a1ffc05bcde375cbbe3adfacecd8d53a"
server_key_iv: "c65c24872041c5b7b252f54f8b5e5fcf"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.486774 27733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:34.510710 27733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:34.514391 27733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:34.514573 27733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:34.522348 27733 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.190:46601
I20260811 22:35:34.522662 27885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.190:46601 every 8 connection(s)
I20260811 22:35:34.523515 27733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/data/info.pb
I20260811 22:35:34.524359 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 27733
I20260811 22:35:34.524489 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-0/wal/instance
I20260811 22:35:34.524845 19278 external_mini_cluster.cc:1428] Setting key 8bd5ea71e7c95fe19410f586c6f2ff10
I20260811 22:35:34.527333 27888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:34.533468 27888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.211.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.189@KRBTEST.COM" created.
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:34.551633 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.536629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45574 (local address 127.18.211.190:46601)
0811 22:35:34.537167 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:34.537192 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:34.537383 (+ 191us) server_negotiation.cc:407] Connection header received
0811 22:35:34.538550 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:34.538555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:34.538773 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:34.539009 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:34.539896 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.541309 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.542858 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.543241 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.543366 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.546383 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:34.546392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:34.546407 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:34.546435 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:34.548476 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.548973 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.548983 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.548986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.549135 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.549505 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.549508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.549509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.549948 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:34.550077 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:34.550446 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:35:34.550822 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":158,"server-negotiator.queue_time_us":399,"spinlock_wait_cycles":92032,"thread_start_us":218,"threads_started":1}
I20260811 22:35:34.551631 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.536624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46601 (local address 127.0.0.1:45574)
0811 22:35:34.537162 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.537264 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:34.538226 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:34.539037 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:34.539047 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:34.539454 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:34.539728 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.539743 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.541440 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.541446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.542719 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.542735 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.543423 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.543430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.543576 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.544366 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:34.544395 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:34.546214 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:34.548598 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.548607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.548616 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.548844 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.549291 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.549293 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.549295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.549411 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.550078 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:34.550083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:34.550214 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 22:35:34.550361 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:35:34.550737 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":1,"thread_start_us":182,"threads_started":1}
I20260811 22:35:34.554201 27888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:35:34.555076 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.554734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50918)
0811 22:35:34.554898 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.554998 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:35:34.555280 27769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
I20260811 22:35:34.556188 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.555938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50932)
0811 22:35:34.556052 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.556124 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:35:34.556453 27888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:41285: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab.
Entry for principal kudu/127.18.211.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab.
I20260811 22:35:34.558319 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:41285
--webserver_interface=127.18.211.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab
--principal=kudu/127.18.211.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:46601,127.18.211.189:41285,127.18.211.188:37259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:35:34.605743 27888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } attempt: 1
I20260811 22:35:34.606702 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.606389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50944)
0811 22:35:34.606554 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.606620 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:35:34.607440 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.607244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50960)
0811 22:35:34.607342 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.607386 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:35:34.607587 27888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:41285: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
W20260811 22:35:34.674953 27900 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:35:34.675235 27900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:34.675258 27900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:34.675297 27900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:35:34.675755 27888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } attempt: 2
I20260811 22:35:34.676620 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.676315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50966)
0811 22:35:34.676478 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.676541 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:35:34.677399 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.677142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50980)
0811 22:35:34.677297 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.677342 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:35:34.677546 27888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:41285: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
W20260811 22:35:34.679872 27900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:34.679927 27900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:34.679947 27900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:34.680011 27900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:34.680061 27900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:34.685616 27900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:46601,127.18.211.189:41285,127.18.211.188:37259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.189.keytab
--principal=kudu/127.18.211.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.189:41285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.211.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:34.686697 27900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:34.687729 27900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:34.696518 27916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:34.696552 27912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:34.696593 27914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:34.696776 27900 server_base.cc:1034] running on GCE node
I20260811 22:35:34.697538 27900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:34.698730 27900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:34.699940 27900 hybrid_clock.cc:648] HybridClock initialized: now 1786487734699906 us; error 48 us; skew 500 ppm
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:34.702886 27900 init.cc:377] Logged in from keytab as kudu/127.18.211.189@KRBTEST.COM (short username kudu)
I20260811 22:35:34.703860 27900 webserver.cc:460] Webserver started at http://127.18.211.189:39267/ using document root <none> and password file <none>
I20260811 22:35:34.704452 27900 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:34.704556 27900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:34.704803 27900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:34.707041 27900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data/instance:
uuid: "a6c7b897cfb047608731e1a176842899"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "8c36fe74cc56305fcbe3a55a50597970"
server_key_iv: "bfaa093df5745af1ca874a30029bcc84"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.707651 27900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/wal/instance:
uuid: "a6c7b897cfb047608731e1a176842899"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "8c36fe74cc56305fcbe3a55a50597970"
server_key_iv: "bfaa093df5745af1ca874a30029bcc84"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.711501 27900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:35:34.714087 27934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:34.715590 27900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:35:34.715754 27900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/wal
uuid: "a6c7b897cfb047608731e1a176842899"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "8c36fe74cc56305fcbe3a55a50597970"
server_key_iv: "bfaa093df5745af1ca874a30029bcc84"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.715885 27900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:34.768729 27888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } attempt: 3
I20260811 22:35:34.769668 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.769370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50986)
0811 22:35:34.769526 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.769588 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:35:34.770125 27900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:34.770448 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.770213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50988)
0811 22:35:34.770337 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.770392 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:35:34.770586 27888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.189:41285: Network error: Client connection negotiation failed: client connection to 127.18.211.189:41285: connect: Connection refused (error 111)
I20260811 22:35:34.773435 27900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:34.773648 27900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:34.781505 27900 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.189:41285
I20260811 22:35:34.781826 28034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.189:41285 every 8 connection(s)
I20260811 22:35:34.782612 27900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/data/info.pb
I20260811 22:35:34.785483 28036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:34.786463 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 27900
I20260811 22:35:34.786619 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-1/wal/instance
I20260811 22:35:34.787004 19278 external_mini_cluster.cc:1428] Setting key a61cd45ee67c1a75e1c98f707a73535a
I20260811 22:35:34.789624 28036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.211.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.211.188@KRBTEST.COM" created.
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:34.806494 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.791785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.18.211.190:46601)
0811 22:35:34.791925 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:34.791928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:34.793171 (+ 1243us) server_negotiation.cc:407] Connection header received
0811 22:35:34.794174 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:34.794177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:34.794247 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:34.794361 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:34.795558 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.796110 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.797374 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.797624 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.797748 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.801825 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:34.801829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:34.801831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:34.801859 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:34.803801 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.804382 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.804390 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.804395 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.804503 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.804869 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.804876 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.804881 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.805216 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:34.805364 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:34.806019 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 22:35:34.806202 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:35:34.807097 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.792384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46601 (local address 127.0.0.1:45580)
0811 22:35:34.793017 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.793064 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:34.793957 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:34.794536 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:34.794543 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:34.794924 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:34.795403 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.795416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.796276 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.796279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.797211 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.797226 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.797797 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.797801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.797977 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.798661 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:34.798678 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:34.801610 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:34.803939 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.803945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.803960 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.804251 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.804591 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.804593 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.804594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.804742 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.805355 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:34.805360 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:34.805673 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 22:35:34.805869 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:35:34.806156 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":165,"threads_started":1}
I20260811 22:35:34.808611 28036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
I20260811 22:35:34.818578 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.809171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50994)
0811 22:35:34.809299 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.809315 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:34.809393 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:34.810075 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:34.810078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:34.810268 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:34.810358 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.810363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.811340 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.811346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.811887 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.811895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.812596 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.812600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.812701 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.813282 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:34.813296 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:34.814803 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:34.816835 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.816837 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.816839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.817093 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.817441 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.817458 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.817459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.817540 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.818065 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:34.818070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:34.818176 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 22:35:34.818276 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:35:34.818383 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:35:34.818750 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.809346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.18.211.189:41285)
0811 22:35:34.809730 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:34.809738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:34.809757 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:35:34.809789 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:34.809793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:34.809954 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:34.810099 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:34.810467 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.811221 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.812007 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.812478 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.812597 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.814951 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:34.814962 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:34.814976 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:34.814999 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:34.816728 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.817205 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.817255 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.817258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.817353 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.817645 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.817648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.817649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.817924 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:34.818098 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:34.818377 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:35:34.818549 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":194,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.211.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab.
Entry for principal kudu/127.18.211.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab.
I20260811 22:35:34.820765 19278 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
/tmp/dist-test-taskG_0iNn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:37259
--webserver_interface=127.18.211.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab
--principal=kudu/127.18.211.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.211.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.211.190:46601,127.18.211.189:41285,127.18.211.188:37259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:35:34.820808 28036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:35:34.821596 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.821351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33448)
0811 22:35:34.821461 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.821536 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:35:34.821843 27938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
I20260811 22:35:34.822836 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.822545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33450)
0811 22:35:34.822687 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.822765 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:35:34.823001 28036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:37259: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
I20260811 22:35:34.872282 28036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } attempt: 1
I20260811 22:35:34.873286 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.872910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33466)
0811 22:35:34.873076 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.873142 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:35:34.874013 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.873799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33472)
0811 22:35:34.873910 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.873963 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:35:34.874158 28036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:37259: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
I20260811 22:35:34.913748 27888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } attempt: 4
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
I20260811 22:35:34.924328 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.914524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.18.211.189:41285)
0811 22:35:34.914693 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:34.914699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:34.914720 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:35:34.914840 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:34.914843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:34.914900 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:34.915004 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:34.915835 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.917141 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.918024 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.918400 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.918522 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.920974 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:34.920977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:34.920979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:34.921023 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:34.922642 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.923140 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.923142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.923144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.923229 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:34.923522 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:34.923525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:34.923526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:34.923700 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:34.923775 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:34.924081 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 22:35:34.924184 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:35:34.924350 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.914394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:50996)
0811 22:35:34.914596 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.914615 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:34.914731 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:34.915035 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:34.915037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:34.915217 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:34.915677 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.915684 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.917335 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.917338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.917900 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:34.917909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:34.918519 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:34.918522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:34.918605 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:34.919177 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:34.919191 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:34.920840 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:34.922760 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.922766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.922770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.923029 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.923320 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:34.923323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:34.923325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:34.923427 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:34.923797 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:34.923800 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:34.923915 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 22:35:34.924030 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:35:34.924179 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 22:35:34.925240 27888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:35:34.926007 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.925730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33486)
0811 22:35:34.925871 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.925931 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 22:35:34.926702 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.926477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33502)
0811 22:35:34.926583 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.926637 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:35:34.926841 27888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:37259: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
I20260811 22:35:34.942315 28036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } attempt: 2
I20260811 22:35:34.943135 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.942843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33508)
0811 22:35:34.942985 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.943058 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:35:34.943944 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.943746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33516)
0811 22:35:34.943848 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.943893 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:35:34.944118 28036 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:37259: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
W20260811 22:35:34.950557 28054 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:35:34.950816 28054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:35:34.950839 28054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:35:34.950883 28054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:35:34.954964 28054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:35:34.955022 28054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:35:34.955044 28054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:35:34.955060 28054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:35:34.955075 28054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:35:34.960326 28054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.211.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.211.190:46601,127.18.211.189:41285,127.18.211.188:37259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/krb5kdc/kudu_127.18.211.188.keytab
--principal=kudu/127.18.211.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.211.188:37259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.211.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:35:34.961391 28054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:35:34.962389 28054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:35:34.971064 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:34.971074 28068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:35:34.971071 28061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:35:34.971405 28054 server_base.cc:1034] running on GCE node
I20260811 22:35:34.972129 28054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:35:34.973392 28054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:35:34.974582 28054 hybrid_clock.cc:648] HybridClock initialized: now 1786487734974556 us; error 36 us; skew 500 ppm
Aug 11 22:35:34 dist-test-slave-8hhm krb5kdc[27706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:35:34.977532 28054 init.cc:377] Logged in from keytab as kudu/127.18.211.188@KRBTEST.COM (short username kudu)
I20260811 22:35:34.978562 28054 webserver.cc:460] Webserver started at http://127.18.211.188:39251/ using document root <none> and password file <none>
I20260811 22:35:34.979166 28054 fs_manager.cc:359] Metadata directory not provided
I20260811 22:35:34.979266 28054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:35:34.979537 28054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:35:34.981801 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data/instance:
uuid: "f69da3bd9e7644f282ba9b74d8d3c804"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "f1114da842064f0c845c543598c8a2d1"
server_key_iv: "b76e58ae3ef8d15bf026834172d4f606"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.982436 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/wal/instance:
uuid: "f69da3bd9e7644f282ba9b74d8d3c804"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "f1114da842064f0c845c543598c8a2d1"
server_key_iv: "b76e58ae3ef8d15bf026834172d4f606"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.986023 27888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } attempt: 1
I20260811 22:35:34.986555 28054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:35:34.986807 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.986512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33520)
0811 22:35:34.986651 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.986729 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:35:34.987509 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:34.987287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33526)
0811 22:35:34.987396 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:34.987453 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:35:34.987663 27888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.211.188:37259: Network error: Client connection negotiation failed: client connection to 127.18.211.188:37259: connect: Connection refused (error 111)
I20260811 22:35:34.989259 28080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:35:34.990624 28054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 22:35:34.990772 28054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/wal
uuid: "f69da3bd9e7644f282ba9b74d8d3c804"
format_stamp: "Formatted at 2026-08-11 22:35:34 on dist-test-slave-8hhm"
server_key: "f1114da842064f0c845c543598c8a2d1"
server_key_iv: "b76e58ae3ef8d15bf026834172d4f606"
server_key_version: "encryptionkey@0"
I20260811 22:35:34.990887 28054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:35:35.006981 28054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:35:35.009967 28054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:35:35.010192 28054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:35:35.017484 28054 rpc_server.cc:307] RPC server started. Bound to: 127.18.211.188:37259
I20260811 22:35:35.017779 28182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.211.188:37259 every 8 connection(s)
I20260811 22:35:35.018501 28054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/data/info.pb
I20260811 22:35:35.021494 28183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:35:35.021481 19278 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu as pid 28054
I20260811 22:35:35.021639 19278 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_0iNn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487433619239-19278-0/minicluster-data/master-2/wal/instance
I20260811 22:35:35.022001 19278 external_mini_cluster.cc:1428] Setting key db3b6782682c6526ae767e1fb2e288fb
I20260811 22:35:35.026618 28183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:35.035308 28036 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } attempt: 3
I20260811 22:35:35.038210 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.023417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.18.211.188:37259)
0811 22:35:35.023812 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.023837 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.023871 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:35:35.024650 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.024670 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.025096 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.025446 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.026413 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.027477 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.028717 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.029568 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.029714 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.032589 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.032596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.032608 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.032640 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.034702 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.035167 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.035175 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.035185 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.035318 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.035691 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.035698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.035702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.036268 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.036496 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.036867 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.037154 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":132,"threads_started":1}
I20260811 22:35:35.039371 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.028624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.190:46601 (local address 127.0.0.1:45582)
0811 22:35:35.029025 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:35.029073 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:35.029189 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:35.029563 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:35.029568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:35.030109 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:35.030275 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.030285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.031125 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.031131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.031811 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.031820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.032589 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.032594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.032706 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.033381 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:35.033401 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:35.035094 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:35.037537 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.037545 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.037551 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.037830 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.038152 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.038164 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.038166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.038282 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.038785 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:35.038789 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:35.038879 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 22:35:35.039017 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 22:35:35.039198 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":111,"threads_started":1}
I20260811 22:35:35.039362 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.028718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.18.211.190:46601)
0811 22:35:35.028887 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.028892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.029157 (+ 265us) server_negotiation.cc:407] Connection header received
0811 22:35:35.029316 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.029320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.029385 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.029505 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.030441 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.030986 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.032008 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.032404 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.032557 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.035281 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.035284 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.035286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.035309 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.037429 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.037935 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.037939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.037941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.038042 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.038397 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.038400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.038402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.038629 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.038703 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.039077 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.039189 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:35:35.040992 28183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.189@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
I20260811 22:35:35.045979 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.035953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33546 (local address 127.18.211.188:37259)
0811 22:35:35.036244 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.036251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.036262 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:35:35.036288 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.036290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.036322 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.037199 (+ 877us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.037332 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.038016 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.039043 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.039346 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.039802 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.041730 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.041734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.041736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.041762 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.044211 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.044674 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.044679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.044681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.044766 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.045052 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.045057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.045058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.045341 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.045444 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.045707 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.045827 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":127,"threads_started":1}
I20260811 22:35:35.045964 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.035770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33546)
0811 22:35:35.035900 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:35.035915 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:35.035992 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:35.036741 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:35.036744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:35.036949 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:35.037084 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.037091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.038116 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.038120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.038907 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.038917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.039478 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.039482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.039581 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.040031 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:35.040043 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:35.041582 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:35.044310 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.044313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.044315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.044572 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.044859 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.044863 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.044865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.044955 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.045444 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:35.045448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:35.045547 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 22:35:35.045654 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:35:35.045804 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.190@KRBTEST.COM
I20260811 22:35:35.049659 28036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: Bootstrap starting.
I20260811 22:35:35.050848 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.041735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.18.211.189:41285)
0811 22:35:35.041846 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.041851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.041868 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:35:35.041960 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.041965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.042030 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.042157 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.042581 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.043188 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.044097 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.044360 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.044463 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.047334 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.047337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.047339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.047362 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.048846 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.049344 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.049347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.049349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.049456 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.049793 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.049797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.049798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.050074 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.050151 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.050511 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.050629 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 22:35:35.051304 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.041508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.189:41285 (local address 127.0.0.1:51000)
0811 22:35:35.041829 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:35.041840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:35.041895 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:35.042176 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:35.042180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:35.042361 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:35.042457 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.042462 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.043309 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.043312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.043960 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.043968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.044474 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.044477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.044574 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.045021 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:35.045035 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:35.047218 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:35.048959 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.048964 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.048965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.049256 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.049569 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.049574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.049578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.049677 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.050233 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:35.050235 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:35.050322 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 22:35:35.051018 (+ 696us) client_negotiation.cc:240] Negotiation successful
0811 22:35:35.051150 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266}
I20260811 22:35:35.052147 28183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:35:35.052798 28036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:35.052850 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.041119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.18.211.190:46601)
0811 22:35:35.041249 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.041253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.041325 (+ 72us) server_negotiation.cc:407] Connection header received
0811 22:35:35.041460 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.041463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.041507 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.041587 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.042926 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.043633 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.045022 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.045381 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.045500 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.048992 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.048998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.049004 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.049033 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.050683 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.051197 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.051200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.051202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.051281 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.051578 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.051580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.051582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.051803 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.052029 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.052547 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.052693 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:35:35.053825 28036 log.cc:824] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:35.054826 27888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } attempt: 2
I20260811 22:35:35.056582 28036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: No bootstrap required, opened a new log
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.188@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.211.189@KRBTEST.COM
Aug 11 22:35:35 dist-test-slave-8hhm krb5kdc[27706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.211.190@KRBTEST.COM for kudu/127.18.211.188@KRBTEST.COM
I20260811 22:35:35.061638 28036 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.061924 28036 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c7b897cfb047608731e1a176842899, State: Initialized, Role: FOLLOWER
I20260811 22:35:35.062337 28036 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.063167 28194 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.063617 28194 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:35.064066 28036 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:35.065794 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.052674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33552)
0811 22:35:35.052827 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:35.052840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:35.052918 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:35.054688 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:35.054690 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:35.054887 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:35.054986 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.054993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.055913 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.055916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.056646 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.056660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.057372 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.057376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.057444 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.057956 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:35.057970 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:35.061493 (+ 3523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:35.064098 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064101 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.064103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.064370 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.064668 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064670 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.064672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.064747 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.065201 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:35.065207 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:35.065362 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 22:35:35.065464 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:35:35.065621 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"spinlock_wait_cycles":768}
I20260811 22:35:35.065924 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.054146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.18.211.189:41285)
0811 22:35:35.054271 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.054275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.054288 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:35:35.054372 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.054375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.054414 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.054506 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.055550 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.056250 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.057357 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.057669 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.057743 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.060826 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.060830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.060833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.060857 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.062584 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064110 (+ 1526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.064114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.064116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.064210 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064568 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.064572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.064574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.064848 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.065031 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.065632 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.065774 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:35:35.066021 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.055196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.211.188:37259 (local address 127.0.0.1:33566)
0811 22:35:35.055426 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 22:35:35.055441 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:35:35.055511 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:35:35.055970 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:35:35.055973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:35:35.056144 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:35:35.056231 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.056237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.057392 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.057395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.058178 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:35:35.058189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.058895 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.058900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:35:35.059010 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.059491 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:35:35.059511 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:35:35.061276 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:35:35.064349 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.064356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.064611 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.064940 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:35:35.064945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:35:35.065035 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:35:35.065548 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:35:35.065552 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:35:35.065649 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:35:35.065731 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:35:35.065863 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260811 22:35:35.066491 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.052727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.18.211.188:37259)
0811 22:35:35.054374 (+ 1647us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.054378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.054391 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:35:35.054430 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.054436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.054482 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.054573 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.055080 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.055792 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.056768 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.057139 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.057271 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.061642 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.061648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.061650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.061696 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.063939 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064482 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.064485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.064487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.064573 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064838 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.064841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.064845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.065088 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.066116 (+ 1028us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.066204 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.066321 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":1576}
I20260811 22:35:35.066640 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.055665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.18.211.188:37259)
0811 22:35:35.055766 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.055770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.055782 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:35:35.055812 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.055815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.055850 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.056344 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.056427 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.057272 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.058347 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.058738 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.059241 (+ 503us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.061701 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.061704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.061707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.061727 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.064228 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.064738 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.064741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.064742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.064818 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.065168 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.065171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.065173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.065425 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.066310 (+ 885us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.066389 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.066508 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":43}
I20260811 22:35:35.070142 27888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182: Bootstrap starting.
I20260811 22:35:35.070173 28183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: Bootstrap starting.
W20260811 22:35:35.070576 28223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:35.070642 28223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:35.072882 28183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:35.072894 27888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182: Neither blocks nor log segments found. Creating new log.
I20260811 22:35:35.073812 27888 log.cc:824] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:35.073810 28183 log.cc:824] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: Log is configured to *not* fsync() on all Append() calls
I20260811 22:35:35.070179 27900 master_runner.cc:186] Error getting master registration for 127.18.211.190:46601: OK, instance_id { permanent_uuid: "0a0edc261e1547588d877ec687311182" instance_seqno: 1786487734520118 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:35:35.074213 27900 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:35:35.075770 19278 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:35:35.076589 28183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: No bootstrap required, opened a new log
I20260811 22:35:35.077044 27824 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0a0edc261e1547588d877ec687311182"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45588
I20260811 22:35:35.079074 27888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182: No bootstrap required, opened a new log
I20260811 22:35:35.080730 28183 raft_consensus.cc:348] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.080991 28183 raft_consensus.cc:729] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69da3bd9e7644f282ba9b74d8d3c804, State: Initialized, Role: FOLLOWER
I20260811 22:35:35.081408 28183 consensus_queue.cc:260] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.082231 27888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.082454 27888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0edc261e1547588d877ec687311182, State: Initialized, Role: FOLLOWER
I20260811 22:35:35.082651 28232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.082782 27888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.082912 28183 sys_catalog.cc:564] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:35.083052 28232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:35.084036 27888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:35:35.084112 27824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:35:35.084249 27824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:35.085755 27824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.086750 27824 leader_election.cc:288] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [CANDIDATE]: Term 1 election: Requested vote from peers a6c7b897cfb047608731e1a176842899 (127.18.211.189:41285), f69da3bd9e7644f282ba9b74d8d3c804 (127.18.211.188:37259)
I20260811 22:35:35.087720 27979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a0edc261e1547588d877ec687311182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a6c7b897cfb047608731e1a176842899"
I20260811 22:35:35.087961 27979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:35.089329 27979 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0edc261e1547588d877ec687311182 in term 1.
W20260811 22:35:35.088649 28274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:35.089778 28274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:35.089766 28126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a0edc261e1547588d877ec687311182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f69da3bd9e7644f282ba9b74d8d3c804"
I20260811 22:35:35.089993 28126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:35:35.091249 28126 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0edc261e1547588d877ec687311182 in term 1.
I20260811 22:35:35.091791 27769 leader_election.cc:302] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0edc261e1547588d877ec687311182, f69da3bd9e7644f282ba9b74d8d3c804; no voters:
I20260811 22:35:35.089026 28234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.092396 28234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:35.092494 28234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:35:35.092543 28234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0edc261e1547588d877ec687311182, State: Running, Role: LEADER
I20260811 22:35:35.092784 28234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } }
I20260811 22:35:35.094818 28234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a0edc261e1547588d877ec687311182. Latest consensus state: current_term: 1 leader_uuid: "0a0edc261e1547588d877ec687311182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.096630 28234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:35.098228 27733 master_runner.cc:263] No leader master found from master 0a0edc261e1547588d877ec687311182
I20260811 22:35:35.098295 27733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:35:35.099042 28054 master_runner.cc:263] No leader master found from master f69da3bd9e7644f282ba9b74d8d3c804
I20260811 22:35:35.099133 28054 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:35:35.107113 28303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:35:35.107267 28303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:35:35.107402 28316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:35:35.108407 28316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:35:35.115485 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.101639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.18.211.190:46601)
0811 22:35:35.101779 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.101786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.103053 (+ 1267us) server_negotiation.cc:407] Connection header received
0811 22:35:35.103288 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.103292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.103333 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.103383 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.104408 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.105323 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.106469 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.106774 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.106851 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.111773 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.111778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.111781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.111813 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.113594 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.114085 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.114089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.114091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.114189 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.114487 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.114490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.114492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.114702 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.114816 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.115189 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.115320 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 22:35:35.123859 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.102225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.18.211.189:41285)
0811 22:35:35.102361 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.102366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.105562 (+ 3196us) server_negotiation.cc:407] Connection header received
0811 22:35:35.105969 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.105972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.106031 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.106082 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.108741 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.109583 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.112954 (+ 3371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.113340 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.113420 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.117629 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.117633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.117637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.117667 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.119359 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.119909 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.119912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.119914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.120012 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.122808 (+ 2796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.122812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.122813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.123076 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.123198 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.123556 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.123685 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:35:35.126971 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:35.102594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.18.211.188:37259)
0811 22:35:35.102712 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.102716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.107201 (+ 4485us) server_negotiation.cc:407] Connection header received
0811 22:35:35.107701 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.107705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.107752 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.107802 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.110745 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.111546 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.115480 (+ 3934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:35.115851 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:35.115927 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:35.120355 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:35:35.120359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:35:35.120362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:35:35.120393 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:35:35.122259 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.125478 (+ 3219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.125482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.125484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.125582 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:35:35.125949 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:35:35.125953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:35:35.125955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:35:35.126225 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:35:35.126353 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:35.126687 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 22:35:35.126816 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:35:35.133936 27979 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [term 1 FOLLOWER]: Refusing update from remote peer 0a0edc261e1547588d877ec687311182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:35:35.134951 28126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [term 1 FOLLOWER]: Refusing update from remote peer 0a0edc261e1547588d877ec687311182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:35:35.136133 28234 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:35:35.136916 28313 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:35:35.141131 28232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a0edc261e1547588d877ec687311182. Latest consensus state: current_term: 1 leader_uuid: "0a0edc261e1547588d877ec687311182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.141294 28232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:35.144820 28194 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a0edc261e1547588d877ec687311182. Latest consensus state: current_term: 1 leader_uuid: "0a0edc261e1547588d877ec687311182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.144950 28194 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:35.155491 28234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a0edc261e1547588d877ec687311182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.155638 28234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:35.156859 28194 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a0edc261e1547588d877ec687311182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.156997 28194 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:35.157433 28326 mvcc.cc:205] Tried to move back new op lower bound from 7317453763097624576 to 7317453762944421888. Current Snapshot: MvccSnapshot[applied={T|T < 7317453763097624576}]
I20260811 22:35:35.160370 28334 mvcc.cc:205] Tried to move back new op lower bound from 7317453763097624576 to 7317453762944421888. Current Snapshot: MvccSnapshot[applied={T|T < 7317453763097624576}]
I20260811 22:35:35.162038 28313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a0edc261e1547588d877ec687311182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.162201 28313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182 [sys.catalog]: This master's current role is: LEADER
I20260811 22:35:35.163172 28232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a0edc261e1547588d877ec687311182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0edc261e1547588d877ec687311182" member_type: VOTER last_known_addr { host: "127.18.211.190" port: 46601 } } peers { permanent_uuid: "a6c7b897cfb047608731e1a176842899" member_type: VOTER last_known_addr { host: "127.18.211.189" port: 41285 } } peers { permanent_uuid: "f69da3bd9e7644f282ba9b74d8d3c804" member_type: VOTER last_known_addr { host: "127.18.211.188" port: 37259 } } }
I20260811 22:35:35.163297 28232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:35:35.163511 28316 catalog_manager.cc:1347] Generated new cluster ID: 3d7b8a9b33c94636902857eae059c460
I20260811 22:35:35.163571 28316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:35:35.165119 28331 mvcc.cc:205] Tried to move back new op lower bound from 7317453763097624576 to 7317453762944421888. Current Snapshot: MvccSnapshot[applied={T|T < 7317453763097624576}]
I20260811 22:35:35.178093 28316 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:35:35.180794 28316 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:35:35.193122 28316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a0edc261e1547588d877ec687311182: Generated new TSK 0
I20260811 22:35:35.194159 28316 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:35:35.223428 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.220880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.18.211.189:41285)
0811 22:35:35.221021 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.221027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.222317 (+ 1290us) server_negotiation.cc:407] Connection header received
0811 22:35:35.222519 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.222523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.222573 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.222626 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.223286 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51024: BlockingRecv error: recv got EOF from 127.0.0.1:51024 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:35:35.224867 27895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.221450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.18.211.190:46601)
0811 22:35:35.221562 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.221566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.223950 (+ 2384us) server_negotiation.cc:407] Connection header received
0811 22:35:35.224121 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.224124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.224174 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.224228 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.224743 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45600: BlockingRecv error: recv got EOF from 127.0.0.1:45600 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 22:35:35.226240 28188 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.221390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33586 (local address 127.18.211.188:37259)
0811 22:35:35.221528 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.221536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.225351 (+ 3815us) server_negotiation.cc:407] Connection header received
0811 22:35:35.225519 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.225522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.225573 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.225623 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.226120 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33586: BlockingRecv error: recv got EOF from 127.0.0.1:33586 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 22:35:35.233227 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.231601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51040 (local address 127.18.211.189:41285)
0811 22:35:35.231750 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.231755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.232223 (+ 468us) server_negotiation.cc:407] Connection header received
0811 22:35:35.232446 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.232450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.232525 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.232579 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.233100 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51040: BlockingRecv error: recv got EOF from 127.0.0.1:51040 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 22:35:35.238310 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.236791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.18.211.189:41285)
0811 22:35:35.236888 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.236892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.237447 (+ 555us) server_negotiation.cc:407] Connection header received
0811 22:35:35.237621 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.237625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.237684 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.237739 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.238212 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51054: BlockingRecv error: recv got EOF from 127.0.0.1:51054 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 22:35:35.243705 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.242245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.18.211.189:41285)
0811 22:35:35.242341 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.242345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.242856 (+ 511us) server_negotiation.cc:407] Connection header received
0811 22:35:35.243030 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.243033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.243080 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.243133 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.243593 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51066: BlockingRecv error: recv got EOF from 127.0.0.1:51066 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 22:35:35.251330 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.249745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.18.211.189:41285)
0811 22:35:35.249884 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.249888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.250417 (+ 529us) server_negotiation.cc:407] Connection header received
0811 22:35:35.250603 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.250606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.250668 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.250719 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.251226 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51076: BlockingRecv error: recv got EOF from 127.0.0.1:51076 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:35:35.264631 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.262754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51090 (local address 127.18.211.189:41285)
0811 22:35:35.262880 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.262884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.263572 (+ 688us) server_negotiation.cc:407] Connection header received
0811 22:35:35.263807 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.263811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.263868 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.263937 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.264508 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51090: BlockingRecv error: recv got EOF from 127.0.0.1:51090 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:35:35.285705 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.283537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.18.211.189:41285)
0811 22:35:35.283679 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.283684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.284437 (+ 753us) server_negotiation.cc:407] Connection header received
0811 22:35:35.284702 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.284706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.284768 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.284832 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.285555 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51094: BlockingRecv error: recv got EOF from 127.0.0.1:51094 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:35:35.321882 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.320465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.18.211.189:41285)
0811 22:35:35.320625 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.320630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.320840 (+ 210us) server_negotiation.cc:407] Connection header received
0811 22:35:35.320971 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.320974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.321034 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.321137 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.321749 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51110: BlockingRecv error: recv got EOF from 127.0.0.1:51110 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 22:35:35.390322 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.389200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.18.211.189:41285)
0811 22:35:35.389407 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.389414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.389434 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:35:35.389502 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.389506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.389570 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.389662 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.390183 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51126: BlockingRecv error: recv got EOF from 127.0.0.1:51126 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 22:35:35.538822 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.521926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.18.211.189:41285)
0811 22:35:35.522076 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.522081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.537888 (+ 15807us) server_negotiation.cc:407] Connection header received
0811 22:35:35.537967 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.537970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.538039 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.538116 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.538686 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51128: BlockingRecv error: recv got EOF from 127.0.0.1:51128 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 22:35:35.798342 28050 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:35:35.797069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51134 (local address 127.18.211.189:41285)
0811 22:35:35.797297 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:35.797300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:35.797464 (+ 164us) server_negotiation.cc:407] Connection header received
0811 22:35:35.797536 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:35.797539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:35.797595 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:35.797688 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:35:35.798196 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51134: BlockingRecv error: recv got EOF from 127.0.0.1:51134 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 22:35:36.072036 28223 catalog_manager.cc:1259] Loaded cluster ID: 3d7b8a9b33c94636902857eae059c460
I20260811 22:35:36.072122 28223 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: loading cluster ID for follower catalog manager: success
I20260811 22:35:36.073936 28223 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: acquiring CA information for follower catalog manager: success
I20260811 22:35:36.075147 28223 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6c7b897cfb047608731e1a176842899: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:35:36.091158 28274 catalog_manager.cc:1259] Loaded cluster ID: 3d7b8a9b33c94636902857eae059c460
I20260811 22:35:36.091253 28274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: loading cluster ID for follower catalog manager: success
I20260811 22:35:36.092916 28274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: acquiring CA information for follower catalog manager: success
I20260811 22:35:36.094110 28274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f69da3bd9e7644f282ba9b74d8d3c804: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:35:36.316556 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:36.312482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.18.211.189:41285)
0811 22:35:36.312729 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:36.312732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:36.312815 (+ 83us) server_negotiation.cc:407] Connection header received
0811 22:35:36.312962 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:36.312965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:36.313102 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:36.313311 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:36.313592 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.314170 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.315048 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.315275 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.315352 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:36.315660 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:36.316116 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:36.316227 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:36.316356 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 22:35:36.316420 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260811 22:35:36.323521 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:36.319932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33590 (local address 127.18.211.188:37259)
0811 22:35:36.320211 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:36.320215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:36.320229 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:35:36.320329 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:36.320331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:36.320445 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:36.320554 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:36.320761 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.321416 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.322239 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.322453 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.322526 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:36.322783 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:36.323212 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:36.323305 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:36.323362 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:35:36.323408 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":76,"threads_started":1}
I20260811 22:35:36.331097 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:36.326591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.18.211.189:41285)
0811 22:35:36.326852 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:36.326857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:36.327153 (+ 296us) server_negotiation.cc:407] Connection header received
0811 22:35:36.327357 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:36.327363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:36.327421 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:36.327474 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:36.327881 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.328609 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.329737 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.329973 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.330097 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:36.330597 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:36.330680 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:36.330728 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:36.330898 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:35:36.330952 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260811 22:35:36.331609 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:36.326592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45610 (local address 127.18.211.190:46601)
0811 22:35:36.326867 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:36.326872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:36.326889 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:35:36.326991 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:36.326994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:36.327168 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:36.327301 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:36.327562 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.328428 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.329630 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.329984 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.330233 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:36.330661 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:36.331031 (+ 370us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:36.331164 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:36.331254 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 22:35:36.331327 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
I20260811 22:35:36.331732 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:35:36.327074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33604 (local address 127.18.211.188:37259)
0811 22:35:36.327207 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:35:36.327213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:35:36.327345 (+ 132us) server_negotiation.cc:407] Connection header received
0811 22:35:36.327513 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:35:36.327519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:35:36.327580 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:35:36.327653 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:35:36.328717 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.329171 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.330555 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:35:36.330765 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:35:36.330869 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:35:36.331143 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:35:36.331228 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:35:36.331297 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:35:36.331521 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 22:35:36.331570 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:35:36.338094 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 27733
I20260811 22:35:36.351418 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 27900
I20260811 22:35:36.370994 19278 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_0iNn/build/debug/bin/kudu with pid 28054
2026-08-11T22:35:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3485 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6932 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (302748 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST